SUMO community discussions

mining sumo weekly metrics

  1. I have a short blog on this subject under Mining SuMo Metrics

    I think the metrics deserves more attention than it currently has. Especially in view of the fast turn-arounds ahead of us. To quote the conclusion:

    ...the answers are there, but what are the questions? This walk-through can just hint at the possibilities that could make SuMo serve our customers even better.
    

    Read it and comment it.

    TiA

    smo

    PS: my old handle (smo) is gone (!?) Has anybody found it;)?

    I have a short blog on this subject under [http://vitosmo.tumblr.com/post/5489756468/mining-sumo-weekly-metrics Mining SuMo Metrics] I think the metrics deserves more attention than it currently has. Especially in view of the fast turn-arounds ahead of us. To quote the conclusion: ...the answers are there, but what are the questions? This walk-through can just hint at the possibilities that could make SuMo serve our customers even better. Read it and comment it. TiA smo PS: my old handle (smo) is gone (!?) Has anybody found it;)?

    Modified by smo on

  2. You focused on support forum metrics, but users go to the support forum if they don't have a quick answer to their first basic search. So I think we must focus first on:

    • top search terms: do they give at least one pertinent article as search results? It is about properly chosen keywords.
    • helpfulness of top viewed articles: maybe there is something missing in some articles.
    You focused on support forum metrics, but users go to the support forum if they don't have a quick answer to their first basic search. So I think we must focus first on: * ''top search terms'': do they give at least one pertinent article as search results? It is about properly chosen keywords. * ''helpfulness of top viewed articles'': maybe there is something missing in some articles.
  3. Interesting blog, graphs often display data better than spreadsheets.

    There appear to be two obvious facts from reading the blog

    1. there are large gaps in data
    2. there is very low performance on the Support forum for answering questions
    The weekly traffic of 3M ends only in 4 per thousand cases in a question.

    So I read that as meaning >99% of visitors do not interact with the SUMO in a way that is recorded

    • is there any other data about this vast majority, e.g. from any other surveys
      • are they visiting and not finding any benefit from the site
      • or are they finding some answer or link but not logging on and commenting or voting
      • do we have any way of seeing how many of those then follow links to KB articles

    So from the remaining minuscule <1%
    ( and what follows is my guestimate from glancing at the blog charts)
    general results seem to suggest

    • only about 15% (of that <1%) get any answer deemed helpful or solving the question
      ( Some of us think there is a problem with answers being marked helpful, that possibly would be better marked solved (see also).)
    • about 30% (of that <1%) get no reply whatsoever
    • about 30% (of that <1%) get a reply that no one marked as helpful
    • ( about 25% apparently missing, what is that?
      all the replies are marked as none helpful)
    • and another data snippet Helpful and Not helpful buttons not really helpful
    As of this morning, the Solved stats vs total number of postings for 4 contributors:
    TonyE = 11.1%
    the-edmeister = 8.0%
    Xircal = 7.9%
    cor-el = 6.7%
    I only did the stats for the 1st four listed in the Top Contributors 


    (I was sure I had already posted a reply, maybe I closed it down inadvertently after preview, or is something odd happening with SUMO, I did get an error message from the site earlier today, maybe it is with the lost handle ! )

    Interesting blog, graphs often display data better than spreadsheets. There appear to be two obvious facts from reading the blog # there are large gaps in data # there is very low performance on the Support forum for answering questions <blockquote>The weekly traffic of 3M ends only in 4 per thousand cases in a question.</blockquote> So I read that as meaning >'''99% of visitors do not interact''' with the SUMO in a way that is recorded * is there any other data about this vast majority, e.g. from any other surveys ** are they visiting and not finding any benefit from the site ** or are they finding some answer or link but not logging on and commenting or voting ** do we have any way of seeing how many of those then follow links to KB articles So from the '''remaining minuscule <1% ''' <br/> ( and what follows is my guestimate from glancing at the blog charts) <br/>general results seem to suggest * only about 15% (of that <1%) get any answer deemed helpful or solving the question <br/>( Some of us think there is a problem with answers being marked helpful, that possibly would be better marked solved [/forums/contributors/704952?page=2#post-36015 (see also)].) * about 30% (of that <1%) get no reply whatsoever * about 30% (of that <1%) get a reply that no one marked as helpful * ( about 25% apparently missing, what is that? <br/> all the replies are marked as none helpful) * and another [/forums/contributors/706289?last=35914#post-35914 data] snippet ''Helpful and Not helpful buttons not really helpful'' <pre>As of this morning, the Solved stats vs total number of postings for 4 contributors: TonyE = 11.1% the-edmeister = 8.0% Xircal = 7.9% cor-el = 6.7% I only did the stats for the 1st four listed in the Top Contributors </pre> (I was sure I had already posted a reply, maybe I closed it down inadvertently after preview, or is something odd happening with SUMO, I did get an error message from the site earlier today, maybe it is with the lost handle ! )
  4. Several problems with the worksheet https://spreadsheets.google.com/ccc?key=piA-a-dXCL2p7vB5pTu0HKA&hl=en#gid=0 :

    1. The "Change" column on the worksheet has been incorrect since 11/5-11/11/2010 because the formula in column C ("Change") has compared column Z to column AA since that time.
    2. If you want to not display #DIV/0! in column C, you must test to be sure neither of the two columns being compared is zero/blank with a formula: IF(OR(D2=0,E2=0),"",(D2-E2)/E2)
      • in the above formula "d2=0" and "e2=0" are "=zero"
      • the result in column C will be blank if either of the compared columns is zero
    Several problems with the worksheet https://spreadsheets.google.com/ccc?key=piA-a-dXCL2p7vB5pTu0HKA&hl=en#gid=0 : #The "Change" column on the worksheet has been incorrect since 11/5-11/11/2010 because the formula in column C ("Change") has compared column Z to column AA since that time. #If you want to not display '''''#DIV/0!''''' in column C, you must test to be sure neither of the two columns being compared is zero/blank with a formula: '''IF(OR(D2=0,E2=0),"",(D2-E2)/E2)''' #*in the above formula "d2=0" and "e2=0" are "='''''zero'''''" #*the result in column C will be blank if either of the compared columns is zero

    Modified by Helper7677 on

  5. Helper7677 said

    Several problems with the worksheet ... column C

    I don't use that column and only have it because it's convenient when there's a massive spike/drop in traffic to see how it distributes across the components of SUMO. It's not worth updating every week (it gets pushed back every time we add a column) or making a more complex formula for.

    John99 said

    Interesting blog, graphs often display data better than spreadsheets. There appear to be two obvious facts from reading the blog
    1. there are large gaps in data
    2. there is very low performance on the Support forum for answering questions

    Yeah, it's again not worth time backfilling data especially since the underlying way we gather that data changes. As for forum performance, we're debating that elsewhere.

    ''Helper7677 [[#post-36215|said]]'' <blockquote> Several problems with the worksheet ... column C </blockquote> I don't use that column and only have it because it's convenient when there's a massive spike/drop in traffic to see how it distributes across the components of SUMO. It's not worth updating every week (it gets pushed back every time we add a column) or making a more complex formula for. ''John99 [[#post-36016|said]]'' <blockquote> Interesting blog, graphs often display data better than spreadsheets. There appear to be two obvious facts from reading the blog # there are large gaps in data # there is very low performance on the Support forum for answering questions </blockquote> Yeah, it's again not worth time backfilling data especially since the underlying way we gather that data changes. As for forum performance, we're debating that elsewhere.
  6. John99 said

    The weekly traffic of 3M ends only in 4 per thousand cases in a question.

    So I read that as meaning >99% of visitors do not interact with the SUMO in a way that is recorded

    • is there any other data about this vast majority, e.g. from any other surveys
      • are they visiting and not finding any benefit from the site
      • or are they finding some answer or link but not logging on and commenting or voting
      • do we have any way of seeing how many of those then follow links to KB articles

    Lately (since the F1 shortcut dropped from Fx 4 and higher) we've been seeing on the order of 2M visitors/week. Of those, about 500,000 a week visit sumo articles (in English). As a whole, they get rated helpful about 66% of the time. I would expect that we get somewhere on the order of another 500,000 visits on localized articles. I also think LOTS of visits are driven by Google which often lead people to individual forum threads. I would imagine that the vast majority of those people would not create accounts and comment.

    As far as seeing how many forum visitors follow links to KB articles — we're working on ways to measure that.

    ''John99 [[#post-36016|said]]'' <blockquote> <blockquote>The weekly traffic of 3M ends only in 4 per thousand cases in a question.</blockquote> So I read that as meaning >'''99% of visitors do not interact''' with the SUMO in a way that is recorded * is there any other data about this vast majority, e.g. from any other surveys ** are they visiting and not finding any benefit from the site ** or are they finding some answer or link but not logging on and commenting or voting ** do we have any way of seeing how many of those then follow links to KB articles </blockquote> Lately (since the F1 shortcut dropped from Fx 4 and higher) we've been seeing on the order of 2M visitors/week. Of those, about 500,000 a week visit sumo articles (in English). As a whole, they get rated helpful about 66% of the time. I would expect that we get somewhere on the order of another 500,000 visits on localized articles. I also think LOTS of visits are driven by Google which often lead people to individual forum threads. I would imagine that the vast majority of those people would not create accounts and comment. As far as seeing how many forum visitors follow links to KB articles — we're working on ways to measure that.