As many of you know, searching for data in Targetprocess can sometimes be more time consuming than expected. Since we would all like to save some time (and search more effectively), we have reworked our global search to make it faster and more reliable.
We plan to start rolling out this new search for customers in the near future. It will be enabled gradually, because it will take some time to re-index all the data. In this regard, the new search will not be immediately available for everyone. Stay tuned. After passing all the checks, we will turn it on for every account.
Search engine replacement
Let me unveil the main change we've made to search. We deprecated our old search engine and built a new one from scratch using Elasticsearch. We chose Elasticsearch because they are a leader in the search engine market. Their engine allows us to improve results for common search tasks, such as relevant search, filtering, aggregation, etc. Replacing the old engine with a mature one simplifies any development related to search, allowing us to move faster. It will now be much easier to implement new features, such as searching by attachments, advanced filtering, and custom orderings.
Relevance of search results
As with many full text search engines, results in Elasticsearch are ordered by relevance by default. We followed the same direction, so search results in Targetprocess will now be ordered by relevance. But, as usual, the devil is in the details.
Relevance algorithm (for those who are hungry for details)
Elasticsearch uses the Term Frequency/Inverse Document Frequency (TF/IDF) algorithm for scoring document relevance.
Once we have a subset of matching documents (entities in our case), they need to be sorted by relevance. Some documents contain all the terms, while others don't. It's important to note that some terms are more important than others. The relevance score of the whole entity depends on the weight of each query term that appears in that document.
So, relevance score is determined by three factors:
2. Inverse document frequency. How often does the term appear in all documents in the collection? The more often, the lower the weight.
Results view redesign
First, you will notice the redesigned search results view. We tried to make it more visual. Comments now are shown in search results (if you search for keywords found inside them) in a tree-style manner. This makes the search results view more consistent with our entity details view, and makes it easier to quickly scan through results. Any keywords found are highlighted, both in comments and entity fields.
Moreover, if the searched keywords are found in Custom Fields (which are all searchable now), they are also highlighted on the results view.
We've also taken some steps to improve the user experience of anyone who uses the QA area in Targetprocess. From now on, when searched keywords are mentioned in test steps, they will appear in the test case result view in a tree-style manner, similar to comments.
Thanks to the changes mentioned, the results view has become more informative and allows you to easily recognize the entity you wanted to find. If the displayed meta data isn't enough, you can open the entity details view by clicking on either the 'Name' or '#ID' of the entity. Due to the fact that we improved navigation back to the search results (they are actually cached), it's now extremely fast and should save a bit of time.
Search by itself just got better. But what if your searched keywords yield too many results, and you cannot find what you're looking for? Sure, you can go through all the pages and eventually discover what you wanted to find, but there should be a better way. For this, you can use filters.
Filters allow you to narrow your search results. You can find the filters area to the left of the result list on the search view.
In many cases you are interested in only one or several types of entity. For this, you can just use the 'ENTITY TYPE' filter and only desired type of entities will be searched among. The filter allows you to select one, several and all types and even have it's own search for faster navigation to the target type.
If you know which states your target entity could be in, use the 'ENTITY STATE' filter, which is identical to the type filter functionality.
It's worth mentioning that type and state filters are dependent on each other. Once you select some types, you can only choose states that are relevant for the for the selected types, and vice versa.
In some other real situation, you may want to find entities that are either hot (were created during the past month) or completely archaic. Just use the brand new 'CREATE DATE' filter. It signals search to take into account only entities that were created during the selected period.
As it was before, the 'All Projects and Teams' checkbox is still present in the filters area and available for usage. It is now enabled by default. Uncheck it to search through currently selected Projects and Teams. Otherwise entities from all Projects and Teams available for you will be considered. We assume that global search should behave globally by default, while narrowing should be an option, not a preset setting.
We did our best to make search relevant and useful right out of the box. In most cases, you won't need to take any steps beyond writing several keywords in the search input. If needed, search can be fine-tuned to solve the most exacting cases. These are cases where the advanced grammar of search can help you get things done. The following table describes a bunch of special characters which can be used in the search input:
|+||big + elephant|
"big elephant" + "small giraffe"
|Signifies AND operation. Searches for entities with both words included (or for phrases, if enclosed in quotes).|
||||elephant | giraffe||Signifies OR operation. Searches for entities with at least one of these words included.|
|""||"Exact phrase"||Exact match. Finds all entities which contain the exact phrase from the search query.|
|*||eleph*||Search by prefix. Finds all words starting with 'eleph' (slows down searching significantly!).|
|After a word, this signifies edit distance (fuzziness). Suppresses emphasizing of mistakes on the search string (slows down searching significantly!).|
After a phrase, this signifies slop amount. Allows other words to appear between words in the phrase (slows down searching significantly!).
Global Search scans the following fields of Targetprocess entities: 'Name', 'Description', and text from all 'Custom Fields'. Searching by Tags, Time Descriptions, and Attachments is not yet supported yet, nor is searching through entities in inactive Projects.
Compared to the old search, we have added a couple of new search sources. So, it's now possible to search among Projects and Teams.