I used to have a blog on the Cambridge Chemistry web server but I tailed off 2 years ago about 13 years of blogging - some chemistry, some polemics about Open Publishing. Suuprt for new equitable models of publishing and much criticism of current scholarship. I coined the term "Publisher-Academic Complex" to describe how academia and the publishing industry fuel each other in a narrow self-centered relationship which disadvantages citizens. That is still the case, maybe even worse.
Knowledge is key to our current world, and much of its generation is publicy funded. That should be available to you, the citizen, but it isn't. For example:
- 85% of the articles published by Taylor and Francis journals are closed. Unless you're in a rich University to have to pay about 40 uSD.
- A paper in 1983 predicted the Ebola outbreak in Liberia. But it was paywalled and obscure. The Liberian Government was outraged.
Yes, We Were Warned About Ebola
By Bernice Dahn, Vera Mussah and Cameron Nutt
April 7, 2015
https://www.nytimes.com/2015/04/08/opinion/yes-we-were-warned-about-ebola.html
This is a major part of the philosophy of this site - that there is vital knowledge already out there. All we have to do is find it and reuse it. And get past publisher paywalls.
The idea is simple. If we search the literature for articles containing the words
- "Liberia" "Gambia" "Sierra Leone" and also
- "Ebola" "Marburg" (highly infective viruses)
Then we would have located this paper.
It's as simple as that.
Todays tweets. Is there a better way of scraping them?
7m Replying to @petermurrayrust @AdamSci12 TBF, I can see the json format being useful too (e.g. I could use it right away without having to deal with harvesting and converting full texts - which reflects my skill levels :-). But less so to fit in an existing workflow around html/xml.
Reply Retweet Like More options petermurrayrust's avatar Peter Murray-Rust @petermurrayrust 30m Replying to @MsPhelps @AdamSci12 Yes, It's horribly fragmented (sigh - 2m distance) EuropePMC is so wonderful by comparison. If I use CORD19 I shall convert it into a useful format.
Reply Retweet Like More options MsPhelps's avatar Biⓐnca Kramer @MsPhelps 59m Replying to @MsPhelps @petermurrayrust @AdamSci12 For your use case, you should be able to replace the PMC subset from EuropePMC. The Elsevier subset should also be available from PMC/EuropePMC at some point in the near future, as I understand. Which leaves the remaining WHO records and the bioRxiv/medRxiv ones?
Reply Retweet Like More options DaveFernig's avatar Dave Fernig @DaveFernig 1h Replying to @petermurrayrust We wasted 2 months. Friday RT-PCR instruments form Uni labs sent to test centres. We could have been abreast of Covid19, rather than 2 deadly weeks behind. More like political idiocy, than lack of will.
Reply Retweet Like More options MsPhelps's avatar Biⓐnca Kramer @MsPhelps 1h Replying to @petermurrayrust @AdamSci12 Indeed, the full-text subsets (fragmented according to license, sigh) comprise 29K papers, the metadata file has metadata for 44K.
My interpretation is that the 44K is for total sourced from the diverse sources, of which a subset is available for full-text download/TDM.
Reply Retweet Like More options petermurrayrust's avatar Peter Murray-Rust @petermurrayrust 1h All academic disciplines are relevant to COVID19. When a academic wakes up s/he shold say "what can I and my discipline do that saves lives?" And let's measure value by public contribution, not citations.
Aparna Nair @DisabilityStor1 Seeing privileged people in academia talk about productivity at a time like this is breathtaking. I’m worried my severely immunocompromised parents in India will contract COVID. That I won’t get to see them. Care for them. That the worst might happen. Get a grip, PLEASE.
Reply Retweet Like More options petermurrayrust's avatar Peter Murray-Rust @petermurrayrust 1h I am sure we have the diagnostic technology. We lack the political will. I am ashamed at how little the UK has done publicly.
Glyn Moody @glynmoody why isn't UK doing this? twitter.com/tveitdal/statu…
twitter.com/glynmoody/status/1241736442038157312
Reply Retweet Like More options petermurrayrust's avatar Peter Murray-Rust @petermurrayrust 1h Great idea, We have the same in our bit of #cambridge
Jen Robinson @suigenerisjen I hope #COVIDー19 inspires #viralkindness like this everywhere. A note in my letterbox today from lovely people on my street in London with numbers to call for help. Just called them to say thank you - and to volunteer to help when I’m back from Aus. More of this in the world.
Reply Retweet Like More options petermurrayrust's avatar Peter Murray-Rust @petermurrayrust 1h Replying to @AdamSci12 Thanks. I've looked at - 15 cols - normal biblio metadata with some additional licence information. Not all papers ( have full text.
Reply Retweet Like More options hjoseph's avatar Heather Joseph @hjoseph 1h Replying to @petermurrayrust @robertkiley and 2 others Peter: also emailed you directly.
Reply Retweet Like More options hjoseph's avatar Heather Joseph @hjoseph 2h Replying to @petermurrayrust @robertkiley and 2 others Yes you should be able to download everything...NLM says lots of folks already have..and if any issues you should reach out to CORD-19 folks directly.
Reply Retweet Like More options petermurrayrust's avatar Peter Murray-Rust @petermurrayrust 2h Replying to @10705013 @egonwillighagen @yvonnenobis WOW! Great. I'll be putting up suggested tasks on the site and inviting Github collaborators.
Reply Retweet Like More options greenman023's avatar malcolm mcewen @greenman023 2h Replying to @petermurrayrust unlikely, given how capitalism operates (exploiting the masses for the benefit of the few) that the situation we find ourselves in could hve been prevented. we could tho have anticipated it & put the necessary health care provisions in place thus negating the economic shut down.
Reply Retweet Like More options OaHelper's avatar OAHelper @OaHelper 2h Replying to @webmink @petermurrayrust This was last archived on the internet archive's wayback machine one day ago.
If you want easy access to internet archive's features like save current page now, see oldest / newest snapshot (etc) and you are on macOS - consider installing our sister app otzberg.net/wayback/
Reply Retweet Like More options shananpeters's avatar Shanan Peters @shananpeters 3h Replying to @ultrazool @petermurrayrust and 2 others I think we have a good model in xDD with great agreements from Elsevier, Taylor and Francis, Wiley and others signed by @UWMadison legal via @UWMadison libraries. I don’t know why publishers like Springer, Cambridge, and others are hesitant. Tried and takes so much energy to move
Reply Retweet Like More options AdamSci12's avatar Adam Day @AdamSci12 4h Replying to @petermurrayrust There's a metadata csv file on the kaggle site which I haven't looked at yet, but it might have a lot of what you're looking for already parsed out: kaggle.com/allen-institut… Hope that helps
Reply Retweet Like More options 10705013's avatar Nina Jeliazkova @10705013 4h Replying to @petermurrayrust @egonwillighagen @yvonnenobis I can help (not full time though )
Reply Retweet Like More options markmoby's avatar Mark 🇨🇦 in Madrid @markmoby 4h Replying to @petermurrayrust Back in January when the government instituted a travel ban from China, were called racist and zenophobic, and when the President was under an impeachment trial? That January?
Reply Retweet Like More options petermurrayrust's avatar Peter Murray-Rust @petermurrayrust 4h Thank you @mikegalsworthy your posts are really valuable
Mike Galsworthy @mikegalsworthy Fascinating read in Sunday Times today. We had warnings in Jan & Feb.
China undertook “the most ambitious... agile and aggressive disease containment effort in history.”
Reply Retweet Like More options petermurrayrust's avatar Peter Murray-Rust @petermurrayrust 4h Very valuable compelling animation on the epidemic 2 min. Watch it. The government shiould have done this in January.
I'm going out for a walk but won't touch anybody or anything
Simon Harris TD @SimonHarrisTD Thought carefully about sharing this. It’s pretty blunt but it’s blunt, honest communication we need. Please watch & retweet. We, all of us, not anyone else, will decide through our actions what happens next. Please do this. Please save lives #coronavirus pic.twitter.com/iIHctC6D6X
Reply Retweet Like More options petermurrayrust's avatar Peter Murray-Rust @petermurrayrust 4h Replying to @petermurrayrust @pheromoneEvo @nytimes Off for 8 km #cambridge walk down Cam . Lovely day. I will "social-distance". I will not touch anything I think this is responsible as it keeps me fit and alert Will start project when I return
Reply Retweet Like More options petermurrayrust's avatar Peter Murray-Rust @petermurrayrust 4h Replying to @pheromoneEvo @nytimes Thanks Very useful link about how NYers are walking in isolation Q to #cambridge people: Are there things we could wear or gestures that show we are takling self-isolation seriously. E.g wearing plastic gloves would be a good sign we cared.
1Reply Retweet Like More options riprap007's avatar Nicholas Ripley @riprap007 4h Replying to @deadlyvices @petermurrayrust Or cared if their modelling was right with the 1% assumption. If, for instance the system is so overwhelmed a figure of 3%+ is not unimaginable
Reply Retweet Like More options deadlyvices's avatar Scope Davies @deadlyvices 4h Replying to @petermurrayrust I don't think so much it's the sums they had an issue with. It's whether or not they actually cared how many people ended up dying.
1Reply Retweet Like More options petermurrayrust's avatar Peter Murray-Rust @petermurrayrust 4h No one who has a head for figures could fail to realise that herd immunity and 1% death rate means half a million dead. A child can do the sums
Scope Davies @deadlyvices Well, who'd a-thunk that putting a 'career psychopath' in charge of Government policy could have led to this kind of decision making? twitter.com/johnharris1969…
twitter.com/deadlyvices/status/1241680463086989313
1Reply 1Retweet 1Like More options petermurrayrust's avatar Peter Murray-Rust @petermurrayrust 4h Richard Horton knows what he is talking about. We should have acted in January and the government (and to some extend the BBC Kuensberg) have failed us. People will die. Just look at the graphs for Italy. We're no different
richard horton @richardhorton1 It said it took a study from Imperial to understand the likely burden of COVID-19 on the NHS. But read the first paper we published on COVID-19 on Jan 24. 32% admitted to ITU with 15% mortality. We have wasted 7 weeks. This crisis was entirely preventable. theguardian.com/world/2020/mar…
Reply Retweet Like More options petermurrayrust's avatar Peter Murray-Rust @petermurrayrust 4h More from @richardhorton1 "the science was there in January" Yes. I'm not an epidemiologist but I understood this then. I trusted that the Government was doing the right thing. I wrote publicly when it was clear they weren't
richard horton @richardhorton1 Laura Kuenssberg says (BBC) that, “The science has changed.” This is not true. The science has been the same since January. What has changed is that govt advisors have at last understood what really took place in China and what is now taking place in Italy. It was there to see.
Reply Retweet Like More options petermurrayrust's avatar Peter Murray-Rust @petermurrayrust 5h Really important posts from @richardhorton1 (unthreaded). The UK cold have stopped the epidemic if they's started in January. This delay is inexcusable. It may well mean the difference between life and death for me.
richard horton @richardhorton1 The morning after the dramatic change in strategy to COVID-19 by this govt, I can’t help but feel angry that it has taken almost two months for politicians and even “experts” to understand the scale of the danger from SARS-CoV-2. Those dangers were clear from the very beginning.
Reply Retweet 1Like More options petermurrayrust's avatar Peter Murray-Rust @petermurrayrust 5h Replying to @petermurrayrust @egonwillighagen and 2 others B/ These post_COVID publishers flourish because
- scientists want to publish
- citizens want knowledge
- citizens are increasingly becoming authors and researchers
- public and philianthropic funders realise the value. Current megapublishers cannot react to these requirements
Reply Retweet Like More options petermurrayrust's avatar Peter Murray-Rust @petermurrayrust 5h Replying to @petermurrayrust @egonwillighagen @yvonnenobis A/ A new generation of post-COVID "publishers" is starting to emerge. Their goal will be to make knowledge free to author and free to read for all citizens on the planet. The leading post-COVID publisher is @Wikimedia They have been followed by the #preprints communities
1Reply Retweet Like More options egonwillighagen's avatar Egon Willighⓐgen @egonwillighagen 5h Replying to @petermurrayrust @yvonnenobis "but how will we pay for coffee, a room, a desk, etc"
Reply Retweet 1Like More options petermurrayrust's avatar Peter Murray-Rust @petermurrayrust 5h Replying to @egonwillighagen @yvonnenobis I agree, Their whole machine is now automated and hardwired into owning content. It's a revolution like Wapping for printing. I guess they do not even understand what we are saying. I am probably dismissed as a pirate.
2Reply Retweet 1Like More options petermurrayrust's avatar Peter Murray-Rust @petermurrayrust 5h Replying to @petermurrayrust @CopyrightRuth @UKLACA 9/9 I shall try to summarise these on github.com/petermr/openVi… and en.wikiversity.org/wiki/WikiJourn…
Reply Retweet Like More options egonwillighagen's avatar Egon Willighⓐgen @egonwillighagen 5h Replying to @yvonnenobis @petermurrayrust I doubt it. the big publishers have been rather immune to such pressure.
1Reply Retweet 1Like More options petermurrayrust's avatar Peter Murray-Rust @petermurrayrust 5h Replying to @petermurrayrust @CopyrightRuth @UKLACA 8/9 The world will be different in 6 months time. Many of us in UK will be dead. Which is more important? a) That we honour copyright in a narrow defensive fashion and protect the publishing industry. b) we liberate knowledge in a 21st century form?.
1Reply Retweet Like More options yvonnenobis's avatar Yvonne Nobis @yvonnenobis 5h Replying to @petermurrayrust Can pressure be put on T&F?
1Reply Retweet Like More options petermurrayrust's avatar Peter Murray-Rust @petermurrayrust 5h Replying to @petermurrayrust @CopyrightRuth @UKLACA 7/ I have never been afraid of civil disobedience. We now have a need and a right to all published knowledge. Not just on COVID but all forms of knowledge. We need history papers. C17 plague society is relevant to human behaviour today.
1Reply Retweet Like More options petermurrayrust's avatar Peter Murray-Rust @petermurrayrust 5h Replying to @yvonnenobis 90% of T+F's papers on respirators is closed. 80% for Elsevier Yes I can mine PMC and I do and I'd like some people helping but the vast majority of knowledge is closed
1Reply Retweet Like More options petermurrayrust's avatar Peter Murray-Rust @petermurrayrust 5h Replying to @robertkiley @chrisbird69 and 2 others ... really? Have they switched off their automatic download triggers? Can we download everything? Have they said this?
Reply Retweet Like More options petermurrayrust's avatar Peter Murray-Rust @petermurrayrust 5h Replying to @petermurrayrust @CopyrightRuth @UKLACA 6/ (B) use "illegal" content. If you don't do (A) then (B) will surely happen . It already did. The first collection of COVID papers came from scraping SciHub.
1Reply Retweet Like More options