SUMO community discussions

Improving Search Results

  1. I want to talk about search results, what we're doing to improve them, and, most importantly, how you can help.

    Next week, we're releasing and update that will, we hope, improve the quality of search results. Some of the changes we're making include:

    • Changing which documents match a given search query.
    • Changing how much weight is given to each match based on where the word is.
    • Changing how we rank the documents that do match.

    These changes appear to result in subtle improvements in results, in the tests we've done.

    So, how can you help?

    If you search for something and don't get the results you expect, please comment on this thread with a few bits of information:

    1. The query you used, exactly as you typed it.
    2. The page you were on when you typed in the query.
    3. The operating system and browser version you were using.
    4. The results you expected, but didn't see.
    5. Any results that were further down the list than you expected.

    Improving search result quality is an ongoing, iterative process, that involves both content and algorithmic changes. If we have specific examples of searches that don't work as expected, it gives us good test cases for understanding how we can improve.

    Thanks!

    I want to talk about '''search results''', what we're doing to '''improve''' them, and, most importantly, '''how you can help'''. Next week, we're releasing and update that will, we hope, improve the quality of search results. Some of the changes we're making include: * Changing which documents '''match''' a given search query. * Changing how much '''weight''' is given to each match based on '''where''' the word is. * Changing how we '''rank''' the documents that do match. These changes appear to result in subtle improvements in results, in the tests we've done. So, '''how can you help?''' If you search for something and don't get the results you expect, please comment on this thread with a few bits of information: # The query you used, exactly as you typed it. # The page you were on when you typed in the query. # The operating system and browser version you were using. # The results you expected, but didn't see. # Any results that were further down the list than you expected. Improving search result quality is an '''ongoing, iterative process''', that involves both '''content''' and '''algorithmic''' changes. If we have specific examples of searches that don't work as expected, it gives us good test cases for understanding how we can improve. Thanks!
  2. First example (no synomyms in keywords):

    1. recuperate profile
    2. https://support.allizom.org/en-US/search?fx=1&os=1&q=recuperate+profile
    3. Windows, Firefox 4
    4. Restore bookmarks, passwords and data from an old Firefox profile
    5. No issue

    Second example (term that is both in the title and keywords, that is repeated a lot in the content?):

    1. remembering passwords
    2. https://support.allizom.org/en-US/search?fx=1&os=1&q=remembering+passwords
    3. Windows, Firefox 4
    4. No issue
    5. n°2 and n°4 should have been n°1 and n°2

    Third example (no keywords appropriate to a topic):

    1. language
    2. https://support.allizom.org/en-US/search?fx=1&os=1&q=language
    3. Windows, Firefox 4
    4. No issue
    5. n°3, 5, 7, 9 should have been n°1, 2, 3, 4
    First example (no synomyms in keywords): # recuperate profile # https://support.allizom.org/en-US/search?fx=1&os=1&q=recuperate+profile # Windows, Firefox 4 # [[Recovering important data from an old profile]] # No issue Second example (term that is both in the title and keywords, that is repeated a lot in the content?): # remembering passwords # https://support.allizom.org/en-US/search?fx=1&os=1&q=remembering+passwords # Windows, Firefox 4 # No issue # n°2 and n°4 should have been n°1 and n°2 Third example (no keywords appropriate to a topic): # language # https://support.allizom.org/en-US/search?fx=1&os=1&q=language # Windows, Firefox 4 # No issue # n°3, 5, 7, 9 should have been n°1, 2, 3, 4
  3. Fourth example (searching a generic mobile article on Windows without changing the OS selector):

    1. Will Firefox work on my phone?
    2. https://support.mozilla.com/en-US/search?os=1&fx=1&q=Will+Firefox+work+on+my+phone%3F
    3. Windows, Firefox 4
    4. Will Firefox work on my mobile device?
    5. No issue

    To prevent this behaviour, I added Fx4, Fx3.5-3.6, Mac OS X, Linux, Windows categories to these generic mobile articles:

    Fourth example (searching a generic mobile article on Windows without changing the OS selector): # Will Firefox work on my phone? # https://support.mozilla.com/en-US/search?os=1&fx=1&q=Will+Firefox+work+on+my+phone%3F # Windows, Firefox 4 # [[Will Firefox work on my phone?]] # No issue To prevent this behaviour, I added Fx4, Fx3.5-3.6, Mac OS X, Linux, Windows categories to these generic mobile articles: * [[How do I install Firefox on a mobile device?]] * [[Will Firefox work on my phone?]] * [[How do I sync Firefox between my desktop and mobile?]]
  4. Another example (lot of occurrences of search terms in articles):

    1. Firefox crashes
    2. http://support.mozilla.com/en-US/search?os=1&fx=1&q=firefox+crashes
    3. Windows, Firefox 4
    4. No issue
    5. Troubleshoot Firefox crashes (closing or quitting unexpectedly) show up only at #22 instead of #1

    Can you give more weight to the title/keywords?

    Another example (lot of occurrences of search terms in articles): #Firefox crashes #http://support.mozilla.com/en-US/search?os=1&fx=1&q=firefox+crashes #Windows, Firefox 4 #No issue #[[Firefox crashes]] show up only at #22 instead of #1 Can you give more weight to the title/keywords?
  5. Hmm, that's an interesting case. Title/keywords are already strongly weighted over all the other fields. In my testing, increasing the weight further didn't affect result order.

    "Firefox" is a stop word for us. If I remove "Firefox" from the stop words list, searching for it returns nearly every document on the site (both all the KB and 1000 support forum questions, which is the limit).

    On the other hand, without ignoring "Firefox," the document "Firefox crashes" shows up on the first page (though not the top). It's worth looking at common queries with "firefox" in them and asking whether or not the word "firefox" actually improves results.

    Hmm, that's an interesting case. Title/keywords are already strongly weighted over all the other fields. In my testing, increasing the weight further didn't affect result order. "Firefox" is a stop word for us. If I remove "Firefox" from the stop words list, searching for it returns nearly every document on the site (both all the KB and 1000 support forum questions, which is the limit). On the other hand, without ignoring "Firefox," the document "Firefox crashes" shows up on the first page (though not the top). It's worth looking at common queries with "firefox" in them and asking whether or not the word "firefox" actually improves results.
  6. Is there a way to match a search phrase with titles even if they contain a stop word and then remove stop words for the rest of the searching? I've noticed a few times that searching for the exact name of an article doesn't turn it up in the results (like Firefox Crashes).

    Also, maybe there is something especially weird with Firefox Crashes. If you just search for crashes it shows up after articles for specific crash signatures. How does that happen?

    Is there a way to match a search phrase with titles even if they contain a stop word and then remove stop words for the rest of the searching? I've noticed a few times that searching for the exact name of an article doesn't turn it up in the results (like Firefox Crashes). Also, maybe there is something especially weird with Firefox Crashes. If you just search for [https://support.mozilla.com/en-US/search?fx=2&os=2&q=crashes crashes] it shows up after articles for specific crash signatures. How does that happen?
  7. Is there a way to match a search phrase with titles even if they contain a stop word and then remove stop words for the rest of the searching?

    Not in Sphinx. A stop word list is universal for a particular index, you can't turn it on or off (or use a different one) per field.

    "Firefox" is a very common word in our KB, so the weighting algorithm isn't going to care much about it. If I can get a list of searches containing "Firefox" we can experiment and see if results get better or worse.

    If you just search for crashes it shows up after articles for specific crash signatures. How does that happen?

    I'm not 100% sure, but as a starting point, it's probably worth adding a summary to Troubleshoot Firefox crashes (closing or quitting unexpectedly). It's slightly handicapped in results by not having the word "crash" there.

    <blockquote>Is there a way to match a search phrase with titles even if they contain a stop word and then remove stop words for the rest of the searching?</blockquote> Not in Sphinx. A stop word list is universal for a particular index, you can't turn it on or off (or use a different one) per field. "Firefox" is a very common word in our KB, so the weighting algorithm isn't going to care ''much'' about it. If I can get a list of searches containing "Firefox" we can experiment and see if results get better or worse. <blockquote>If you just search for crashes it shows up after articles for specific crash signatures. How does that happen?</blockquote> I'm not 100% sure, but as a starting point, it's probably worth adding a summary to [[Firefox crashes]]. It's slightly handicapped in results by not having the word "crash" there.
  8. Another example (same reason as in the first example, that is no synomyms in keywords):

    1. how to solve my problem
    2. https://support.mozilla.com/en-US/search?fx=1&q=how+to+solve+my+problem&os=1&page=1
    3. Windows, Firefox 4
    4. Basic troubleshooting and Troubleshoot extensions, themes and hardware acceleration issues to solve common Firefox problems
    5. No issue
    Another example (same reason as in the first example, that is no synomyms in keywords): # how to solve my problem # https://support.mozilla.com/en-US/search?fx=1&q=how+to+solve+my+problem&os=1&page=1 # Windows, Firefox 4 # [[Basic troubleshooting]] and [[Troubleshooting extensions and themes]] # No issue
  9. 4. Troubleshoot and diagnose Firefox problems and Troubleshoot extensions, themes and hardware acceleration issues to solve common Firefox problems

    In this case, neither of those articles contains the word "solve." I would add it to the keywords field for both.

    <blockquote>4. [[Basic Troubleshooting]] and [[Troubleshooting extensions and themes]]</blockquote> In this case, neither of those articles contains the word "solve." I would add it to the keywords field for both.
  10. Alternative spellings for search terms.
    Regardless of which is the correct spelling, especially in different countries, spellings of words may vary. One common variation may be the s or z in word endings such as customize vs customise; possibly it would help if the alternative spelling using an s were to be added as keywords. (do you also make any attempt to add common accidental mis spellings)

    Example using customise

    1. customise
    2. https://support.mozilla.com/en-US/search?q=customise
    3. Windows XP Firefox 3.6.13
    4. not unexpected, but it missed many results, returning only 16 items, none of which were KB items

    Example using customize

    1. customize
    2. https://support.mozilla.com/en-US/search?q=customize
    3. Windows XP Firefox 3.6.13
    4. as expected, compare with results above. This search returns 318 results, including several screens of KB items. First four items, Find and install add-ons to add features to Firefox, Customize Firefox controls, buttons and toolbars, Custom installation of Firefox on Windows & Customizing Firefox

    I have noted improvements in the search.

    However maybe I have recently seen some unexpected entries relating to firefox mobile, when I would have expected only firefox desktop results to be returned, but I have no actual examples of that to quote at the moment.

    Alternative spellings for search terms.<br> Regardless of which is the correct spelling, especially in different countries, spellings of words may vary. One common variation may be the s or z in word endings such as customi'''z'''e vs customi'''s'''e; possibly it would help if the alternative spelling using an s were to be added as keywords. (do you also make any attempt to add common accidental mis spellings) '''Example using customise''' # customise #https://support.mozilla.com/en-US/search?q=customise # Windows XP Firefox 3.6.13 # not unexpected, but it missed many results, returning only 16 items, none of which were KB items '''Example using customize''' # customize #https://support.mozilla.com/en-US/search?q=customize # Windows XP Firefox 3.6.13 # as expected, compare with results above. This search returns 318 results, including several screens of KB items. First four items, [[Customizing Firefox with add-ons]], [[How to customize the toolbar]], [[Custom installation of Firefox on Windows]] & [[Customizing Firefox]] --------- I have noted improvements in the search. However maybe I have recently seen some unexpected entries relating to firefox mobile, when I would have expected only firefox desktop results to be returned, but I have no actual examples of that to quote at the moment.
  11. Example with a compound word:

    1. marque-page (means bookmark in French)
    2. https://support.mozilla.com/fr/search?fx=1&os=1&q=marque-page
    3. Windows, Firefox 4
    4. Same as https://support.mozilla.com/fr/search?os=1&fx=1&q=marque+page
    5. Same as 4.
    Example with a compound word: # marque-page (means bookmark in French) # https://support.mozilla.com/fr/search?fx=1&os=1&q=marque-page # Windows, Firefox 4 # Same as https://support.mozilla.com/fr/search?os=1&fx=1&q=marque+page # Same as 4.
  12. Scoobi, is that a regression, or as far as you know has it never worked?

    Scoobi, is that a regression, or as far as you know has it never worked?
  13. Scoobi, is that a regression, or as far as you know has it never worked?

    I think it has never worked. It gives the results of https://support.mozilla.com/fr/search?fx=1&os=1&q=marque+-page ("marque -page" which means "marque" without "page").

    ''Scoobi, is that a regression, or as far as you know has it never worked?'' I think it has never worked. It gives the results of https://support.mozilla.com/fr/search?fx=1&os=1&q=marque+-page ("marque -page" which means "marque" without "page").
  14. We only started supporting that syntax ("-foo" means "without foo") in the past 3 months. I noticed that searching for "marque\-page" seems to return the expected result set.

    I've gone ahead and filed a bug.

    We only started supporting that syntax ("-foo" means "without foo") in the past 3 months. I noticed that searching for "marque\-page" seems to return the expected result set. I've gone ahead and filed [https://bugzilla.mozilla.org/show_bug.cgi?id=645681 a bug].
  15. Example with a French stop word:

    1. page de démarrage (means "start page" in French)
    2. https://support.mozilla.com/fr/search?q=page+de+d%C3%A9marrage
    3. Windows, Firefox 4
    4. Same as https://support.mozilla.com/fr/search?q=page+d%C3%A9marrage
    5. Same as 4.

    Is it possible to have stop words by locale?

    Example with a French stop word: # page de démarrage (means "start page" in French) # https://support.mozilla.com/fr/search?q=page+de+d%C3%A9marrage # Windows, Firefox 4 # Same as https://support.mozilla.com/fr/search?q=page+d%C3%A9marrage # Same as 4. Is it possible to have stop words by locale?
  16. As asked previously, what about keywords by locale, customise spelt with an s is the spelling used by the UI e.g. when right clicking on a empty area of toolbar in en-GB, and I would imagine the same for Australian English.

    As asked previously, what about keywords by locale, customise spelt with an s is the spelling used by the UI e.g. when right clicking on a empty area of toolbar in en-GB, and I would imagine the same for Australian English.
  17. John99 said

    what about keywords by locale

    Keywords are managed by article, so you're free to add new ones. customise is already a keyword in Find and install add-ons to add features to Firefox and Customize Firefox controls, buttons and toolbars.

    ''John99 [[#post-35866|said]]'' <blockquote> what about keywords by locale </blockquote> Keywords are managed by article, so you're free to add new ones. customise is already a keyword in [[Customizing Firefox with add-ons]] and [[How do I customize the toolbars?]].
  18. We can't do stopwords by locale--the list is global, or at least per-index--but we can add words like `de` to the list.

    We can't do stopwords by locale--the list is global, or at least per-index--but we can add words like `de` to the list.
  19. According to French KB article names, I'd say French basic stop words are: `le`, `la`, `les`, `l`, `un`, `une`, `de`, `du`, `des`, `et`, `ou`, `à`, `au`, `aux`, `en`, `se` (mean in English 'the', 'a', 'one', 'of', 'some', 'and', 'or', 'at', 'in', 'itself').

    According to French KB article names, I'd say French basic stop words are: `le`, `la`, `les`, `l`, `un`, `une`, `de`, `du`, `des`, `et`, `ou`, `à`, `au`, `aux`, `en`, `se` (mean in English 'the', 'a', 'one', 'of', 'some', 'and', 'or', 'at', 'in', 'itself').
  20. Worth adding 's'? Not sure how often "s'" comes up for us.

    Worth adding 's'? Not sure how often "s'" comes up for us.
  1. 1
  2. 2