Project

General

Profile

Actions

Feature #15429

closed

[Workbench] Use trigram based full text search.

Added by Eric Biagiotti over 4 years ago. Updated over 4 years ago.

Status:
Resolved
Priority:
Normal
Assigned To:
Category:
-
Target version:
Story points:
0.5
Release relationship:
Auto

Description

Since we plan on deprecating @@, as an available API list method filter, we should switch Workbench 1 full text search to use the trigram index based ["any", "ilike", "%%"] filter.


Subtasks 1 (0 open1 closed)

Task #15620: Review 15429-wb1-use-trigram-searchResolvedLucas Di Pentima10/15/2019Actions

Related issues

Related to Arvados - Idea #15430: [API] Remove the @@ list method filterResolvedLucas Di Pentima08/16/2021Actions
Related to Arvados Epics - Idea #15333: Workbench2 feature parity with WorkbenchClosed01/01/202012/31/2021Actions
Actions #1

Updated by Eric Biagiotti over 4 years ago

  • Related to Idea #15430: [API] Remove the @@ list method filter added
Actions #2

Updated by Tom Morris over 4 years ago

  • Target version set to To Be Groomed
Actions #3

Updated by Tom Morris over 4 years ago

  • Story points set to 0.5
Actions #4

Updated by Tom Morris over 4 years ago

  • Target version changed from To Be Groomed to Arvados Future Sprints
Actions #5

Updated by Tom Morris over 4 years ago

  • Target version changed from Arvados Future Sprints to 2019-09-25 Sprint
Actions #6

Updated by Lucas Di Pentima over 4 years ago

  • Assigned To set to Lucas Di Pentima
Actions #7

Updated by Lucas Di Pentima over 4 years ago

  • Target version changed from 2019-09-25 Sprint to 2019-10-09 Sprint
Actions #8

Updated by Tom Morris over 4 years ago

  • Target version changed from 2019-10-09 Sprint to 2019-10-23 Sprint
Actions #9

Updated by Lucas Di Pentima over 4 years ago

  • Status changed from New to In Progress
Actions #10

Updated by Lucas Di Pentima over 4 years ago

  • Release set to 22
Actions #11

Updated by Lucas Di Pentima over 4 years ago

Updates at b93675913 - branch 15429-wb1-use-trigram-search
Test run: https://ci.curoverse.com/job/developer-run-tests/1590/

  • Replaced to_tsquery() with to_tsquery_filters() that adds a ["any", "ilike", "%term%"] for every "word" on the user search query.
  • No test updates needed as integration tests don't care how the search is done.

Some clarification needed: On to_tsquery()'s comment there were some cases documented stating that underscore chars should be treated as non-word, but from my manual tests it seems that isn't the case with the current regular expression. Not sure if the comments were old or the code buggy and I should add that feature to the new function.

Actions #12

Updated by Tom Clegg over 4 years ago

Since it's no longer doing tsquery things, to_tsquery_filters() doesn't seem like the best name -- maybe ilike_filters()?

""." is a word char in FT queries" → "...ilike queries"

comment "It returns null if..." → "It returns [] if..." (good change!)

to_tsquery_filters(q).forEach(function(f) { filters.push(f) }) ...could be just filters = filters.concat(to_tsquery_filters(q)) like the line above

Actions #13

Updated by Lucas Di Pentima over 4 years ago

Updates at 13ac0cadc
Test run: https://ci.curoverse.com/job/developer-run-tests/1593/

Addressed the above comments and also renamed the to_tsquery.js file to ilike_filters.js

Actions #15

Updated by Tom Clegg over 4 years ago

Lucas Di Pentima wrote:

Some clarification needed: On to_tsquery()'s comment there were some cases documented stating that underscore chars should be treated as non-word, but from my manual tests it seems that isn't the case with the current regular expression. Not sure if the comments were old or the code buggy and I should add that feature to the new function.

I don't think it matters: the purpose of defining "word chars" in the previous code was to match the postgres full-text query behavior. The new code just has to behave sensibly with "ilike". I suppose we can either treat an underscore as a word char and leave it alone (so it matches any single char) or escape it with a backslash (so it matches a literal underscore). I'd guess that most users will expect it to match a literal underscore, but the wildcard option also seems reasonable.

Actions #16

Updated by Tom Clegg over 4 years ago

LGTM @ 13ac0cadc

Actions #18

Updated by Tom Clegg over 4 years ago

LGTM, thanks

Actions #19

Updated by Lucas Di Pentima over 4 years ago

  • Status changed from In Progress to Resolved
Actions #20

Updated by Lucas Di Pentima about 4 years ago

  • Related to Idea #15333: Workbench2 feature parity with Workbench added
Actions

Also available in: Atom PDF