Is it possible to search the forum for certain 'terms' or 'phrases' or combination of words?

Quill

Jedi
What I mean by that is, that It's been my habit for quite a while to search something on the net using quotation marks around the searched words to narrow down otherwise thousands or millions of results.

An example: if I was to search on the net for info about the book 'myth of sanity', I would enter it like "myth of sanity", to make sure that only certain, included combination of words are included in the search, as the results would otherwise include any and every separate words(or combinations of the same).

I think it might be just the way the forum software works, but is there other ways than using net search engines with certain parameters(which I guess would show only public threads) to search from this site?
 

ToxicCoward

The Force is Strong With This One
You can type: site:cassiopaea.org "myth of sanity" on duckduckgo (or some other search engine).
Good solution, at least for public part of forum.

Why forum search works like that?
Search must be fast and affordable, so some optimizations have to be done.
Imagine that someone wrote post
A lazy dog jumps over the fox
Since words "a", "over", "the" are very common (Most common words in English - Wikipedia) it is not worth to allow to search by them, answers wouldn't be really relevant and You could abuse it by spamming for "a" and asking forum to use all its processing power to check almost every post possible.
So after excluding most common words system keeps index with "lazy", "dog", "jumps", "fox". That way each time You search for "dog" it only checks list of those 4 words instead of doing letter-by-letter comparison on every post possible.

Would it be possible to bring full search to this forums.
Most likely yes, with paid plugins.
I don't think it would be worth purchasing, unless enabled for certain, small group of users, otherwise there would be a risk of people abusing that feature.
 

Scottie

Administrator
Administrator
Moderator
FOTCM Member
The best way to enhance search on XenForo is to use the official Enhanced Search add-on, which uses Elasticsearch. Unfortunately, that requires a 2nd server just to run ES. ES is kind of like "run your own Google for your web site".

Exact text searches are indeed a pain since the most reliable way to do them is with a mysql LIKE query, which is very, VERY slow - especially when it must search through our 1 million posts!
 

ToxicCoward

The Force is Strong With This One
To find key words in transcripts, I use:
"your key word here" session site:cassiopaea.org/forum

To find keywords in transcripts its more comfortable to use this:
With .html version You can just use quick find (CTRL+F) to do fulltext search on all transcripts at once.
 

aimarok

Jedi Master
FOTCM Member
According to Internet Xenforo should use MySQL fulltext search (FTS). FTS is more expensive than LIKE but it has two nice features: relevance calculation and morphology support. I've just searched "full text search" on this forum and the first result link contained only one word from the query, though somewhere at 15th position there was a result with highlighted "full text search" text. So it ain't know a thing about relevancy.

Scottie, does XF require manual activation of FTS and uses LIKE by default?
 

mrtn

Dagobah Resident
Maybe searching with asterisk placeholder helps. It makes for kind of fuzzy searches like 'somethi*'. For example searching 'myth*of*sanity' (without the single quotes) brings up Quill's and others posts from this thread.
 

aimarok

Jedi Master
FOTCM Member
I got XenForo sources and peaked inside. It uses MySQL FTS in BOOLEAN mode. Here is a link to MySQL documentation explaining how to compose correct search queries. XF search index table is of MyISAM type. For example to find exact phrase you need to put it in double quotes, like "full text search". Searching the same phrase without quotes makes XF return all posts with at least one of the phrase words which is not helpful at all.

There is also a rather long list of stop words excluded from searching in XF. For example every word in "everything becomes possible" is in this list.
'a\'s', 'able', 'about', 'above', 'according', 'accordingly', 'across', 'actually',
'after', 'afterwards', 'again', 'against', 'ain\'t', 'all', 'allow', 'allows',
'almost', 'alone', 'along', 'already', 'also', 'although', 'always', 'am',
'among', 'amongst', 'an', 'and', 'another', 'any', 'anybody', 'anyhow',
'anyone', 'anything', 'anyway', 'anyways', 'anywhere', 'apart', 'appear', 'appreciate',
'appropriate', 'are', 'aren\'t', 'around', 'as', 'aside', 'ask', 'asking',
'associated', 'at', 'available', 'away', 'awfully', 'be', 'became', 'because',
'become', 'becomes', 'becoming', 'been', 'before', 'beforehand', 'behind', 'being',
'believe', 'below', 'beside', 'besides', 'best', 'better', 'between', 'beyond',
'both', 'brief', 'but', 'by', 'c\'mon', 'c\'s', 'came', 'can',
'can\'t', 'cannot', 'cant', 'cause', 'causes', 'certain', 'certainly', 'changes',
'clearly', 'co', 'com', 'come', 'comes', 'concerning', 'consequently', 'consider',
'considering', 'contain', 'containing', 'contains', 'corresponding', 'could', 'couldn\'t', 'course',
'currently', 'definitely', 'described', 'despite', 'did', 'didn\'t', 'different', 'do',
'does', 'doesn\'t', 'doing', 'don\'t', 'done', 'down', 'downwards', 'during',
'each', 'edu', 'eg', 'eight', 'either', 'else', 'elsewhere', 'enough',
'entirely', 'especially', 'et', 'etc', 'even', 'ever', 'every', 'everybody',
'everyone', 'everything', 'everywhere', 'ex', 'exactly', 'example', 'except', 'far',
'few', 'fifth', 'first', 'five', 'followed', 'following', 'follows', 'for',
'former', 'formerly', 'forth', 'four', 'from', 'further', 'furthermore', 'get',
'gets', 'getting', 'given', 'gives', 'go', 'goes', 'going', 'gone',
'got', 'gotten', 'greetings', 'had', 'hadn\'t', 'happens', 'hardly', 'has',
'hasn\'t', 'have', 'haven\'t', 'having', 'he', 'he\'s', 'hello', 'help',
'hence', 'her', 'here', 'here\'s', 'hereafter', 'hereby', 'herein', 'hereupon',
'hers', 'herself', 'hi', 'him', 'himself', 'his', 'hither', 'hopefully',
'how', 'howbeit', 'however', 'i\'d', 'i\'ll', 'i\'m', 'i\'ve', 'ie',
'if', 'ignored', 'immediate', 'in', 'inasmuch', 'inc', 'indeed', 'indicate',
'indicated', 'indicates', 'inner', 'insofar', 'instead', 'into', 'inward', 'is',
'isn\'t', 'it', 'it\'d', 'it\'ll', 'it\'s', 'its', 'itself', 'just',
'keep', 'keeps', 'kept', 'know', 'known', 'knows', 'last', 'lately',
'later', 'latter', 'latterly', 'least', 'less', 'lest', 'let', 'let\'s',
'like', 'liked', 'likely', 'little', 'look', 'looking', 'looks', 'ltd',
'mainly', 'many', 'may', 'maybe', 'me', 'mean', 'meanwhile', 'merely',
'might', 'more', 'moreover', 'most', 'mostly', 'much', 'must', 'my',
'myself', 'name', 'namely', 'nd', 'near', 'nearly', 'necessary', 'need',
'needs', 'neither', 'never', 'nevertheless', 'new', 'next', 'nine', 'no',
'nobody', 'non', 'none', 'noone', 'nor', 'normally', 'not', 'nothing',
'novel', 'now', 'nowhere', 'obviously', 'of', 'off', 'often', 'oh',
'ok', 'okay', 'old', 'on', 'once', 'one', 'ones', 'only',
'onto', 'or', 'other', 'others', 'otherwise', 'ought', 'our', 'ours',
'ourselves', 'out', 'outside', 'over', 'overall', 'own', 'particular', 'particularly',
'per', 'perhaps', 'placed', 'please', 'plus', 'possible', 'presumably', 'probably',
'provides', 'que', 'quite', 'qv', 'rather', 'rd', 're', 'really',
'reasonably', 'regarding', 'regardless', 'regards', 'relatively', 'respectively', 'right', 'said',
'same', 'saw', 'say', 'saying', 'says', 'second', 'secondly', 'see',
'seeing', 'seem', 'seemed', 'seeming', 'seems', 'seen', 'self', 'selves',
'sensible', 'sent', 'serious', 'seriously', 'seven', 'several', 'shall', 'she',
'should', 'shouldn\'t', 'since', 'six', 'so', 'some', 'somebody', 'somehow',
'someone', 'something', 'sometime', 'sometimes', 'somewhat', 'somewhere', 'soon', 'sorry',
'specified', 'specify', 'specifying', 'still', 'sub', 'such', 'sup', 'sure',
't\'s', 'take', 'taken', 'tell', 'tends', 'th', 'than', 'thank',
'thanks', 'thanx', 'that', 'that\'s', 'thats', 'the', 'their', 'theirs',
'them', 'themselves', 'then', 'thence', 'there', 'there\'s', 'thereafter', 'thereby',
'therefore', 'therein', 'theres', 'thereupon', 'these', 'they', 'they\'d', 'they\'ll',
'they\'re', 'they\'ve', 'think', 'third', 'this', 'thorough', 'thoroughly', 'those',
'though', 'three', 'through', 'throughout', 'thru', 'thus', 'to', 'together',
'too', 'took', 'toward', 'towards', 'tried', 'tries', 'truly', 'try',
'trying', 'twice', 'two', 'un', 'under', 'unfortunately', 'unless', 'unlikely',
'until', 'unto', 'up', 'upon', 'us', 'use', 'used', 'useful',
'uses', 'using', 'usually', 'value', 'various', 'very', 'via', 'viz',
'vs', 'want', 'wants', 'was', 'wasn\'t', 'way', 'we', 'we\'d',
'we\'ll', 'we\'re', 'we\'ve', 'welcome', 'well', 'went', 'were', 'weren\'t',
'what', 'what\'s', 'whatever', 'when', 'whence', 'whenever', 'where', 'where\'s',
'whereafter', 'whereas', 'whereby', 'wherein', 'whereupon', 'wherever', 'whether', 'which',
'while', 'whither', 'who', 'who\'s', 'whoever', 'whole', 'whom', 'whose',
'why', 'will', 'willing', 'wish', 'with', 'within', 'without', 'won\'t',
'wonder', 'would', 'wouldn\'t', 'yes', 'yet', 'you', 'you\'d', 'you\'ll',
'you\'re', 'you\'ve', 'your', 'yours', 'yourself', 'yourselves', 'zero'
 
Last edited:
Top Bottom