Evaluating Digital Projects… Or why we shouldn’t use social media to evaluate museum projects!
Posted on

Our recent ‘live’ Twitter chat, #DLNETChat on 3 March looked at the topic of evaluating digital projects, and it’s clear that there was a lot to be said on the subject.
We have done our best to compile a Storify of the conversations which took place on the subject – click here for more details. However, here we are picking out some of the particular discussion points which may be of interest.
Our ice-breaker poll launched on Twitter before the chat started, and was contentious in the options which were conspicuously absent:
For today’s (3 Mar) #DLNETChat – what is the best way to evaluate digital engagement projects?
— DLNET (@DLNET) March 3, 2017
Some very good links were shared, even before the ‘live’ chat started…
For today’s #DLNETChat this report by us @flow_global includes summaries of evaluations of digital projects in 3.2 https://t.co/QpMLJPTv8R
— bridgetmck (@bridgetmck) March 3, 2017
Once things got going, John McMahon brought plenty of good advice…
.@heritagelottery‘s ‘Digital tech. in heritage projects’ guidance includes top-level evaluation tips: https://t.co/4RWgRWuaZR #DLNETchat
— John McMahon (@JohnMcArtsEd) March 3, 2017
…But I think the main things are, as for any project – know the impact you want, and how to measure it, from the start; #DLNETchat
— John McMahon (@JohnMcArtsEd) March 3, 2017
Others contributed some good links too:
Hi all on #DLNETChat. We’ve evaluated a number of digital projects + programmes e.g. eval of @pplscollection prog: https://t.co/AaKtm4h67H
— ERS Ltd (@ERS_Limited) March 3, 2017
Re: research/evaluation as part of digital design process. Blue boxes are where research can/could/should? be intergrated #dlnetchat pic.twitter.com/LCMlqNgFIx
— Lindsey Green (@lindsey_green) March 3, 2017
There were also some very interesting side discussions, including one about Sentiment Analysis (click here for a Wikipedia article), which is essentially a way of digitally analysing qualitative data including social media posts and comments…
@DLNET are there other methods that people use e.g. Sentiment analysis?
— Marie Hobson (@mchobson) March 3, 2017
@stuartdberry @mchobson I tried sentiment analysis for social media with different tools but it is not always accurate
— Elena Villaespesa (@elenustika) March 3, 2017
Research on sentiment analysis for social media and cultural orgs #DLNETChat https://t.co/XS1c7P5MX2
— Lindsey Green (@lindsey_green) March 3, 2017
@marthasadie @elenustika @stuartdberry @mchobson I think @EpiphanyLboro was looking at this too as part of PHD. Was it published?
— Lindsey Green (@lindsey_green) March 3, 2017
@lindsey_green @marthasadie @elenustika @stuartdberry @mchobson Yes indeed: https://t.co/yeBmzoGGnN – but ‘not very accurate’ saves time. 🙂
— David Gerrard (@EpiphanyLboro) March 3, 2017
@EpiphanyLboro @lindsey_green @marthasadie @elenustika @stuartdberry @mchobson Sentiment analysis=not accurate. Did JISC funded project 1/2
— Dr Eric Jensen (@JensenWarwick) March 3, 2017
And it very quickly transpired why this might particularly apply to museum audiences especially…
@EpiphanyLboro @lindsey_green @marthasadie @elenustika @stuartdberry @mchobson Social media =skewed museum sample. https://t.co/JhaszBNlY2
— Dr Eric Jensen (@JensenWarwick) March 3, 2017
@mchobson @EpiphanyLboro @lindsey_green @marthasadie @elenustika @stuartdberry Sampling issue: we found 30% in museum tweets work in sector
— Dr Eric Jensen (@JensenWarwick) March 3, 2017
@lindsey_green @mchobson @EpiphanyLboro @marthasadie @elenustika @stuartdberry Museum worker tweeters avoided negative out of career concern
— Dr Eric Jensen (@JensenWarwick) March 3, 2017
@lindsey_green Conclusions about social media data for evaluation. 1) social inclusion. Project website: https://t.co/kJFiwUbcVG @DLNET pic.twitter.com/bAzpi1M9sh
— Dr Eric Jensen (@JensenWarwick) March 3, 2017
@lindsey_green Further Conclusions: methods issues in using social media data for evaluation @DLNET pic.twitter.com/1kuy0f6ueF
— Dr Eric Jensen (@JensenWarwick) March 3, 2017
There were many other useful and interesting conversations, including links to case-studies, tools and methodologies in the chat, so please look at the Storify for the full run-down…
Don’t forget to look out for our next #DLNETChat on the first Friday of the month, and watch our Twitter account for further details!