Disqus - Latest Comments for Aarontay

Re: Guardrails don't fit with scholarship - The Distant Librarian

aarontay — Tue, 29 Jul 2025 20:02:50 -0000

Yep. To be clear it's just the two Clarivate products and possibly Web of Science research assistant that might be affected. So far everything else I briefly tested eg Scopus AI, Scite, Undermind, EBSCO stuff etc all seem fine on first glance.

Re: Deep Dive into Three AI Academic Search Tools

aarontay — Tue, 01 Jul 2025 13:59:33 -0000

My view is relevance is the main thing we should look out for. My sense is given current technology, there likely will be a tradeoff between transparency of search (ie interpretability) and higher relevancy. Most users (save those doing systematic reviews) probably will not care so much the search results aren't as interpretable or explainable as classic keyword search. If they do, they can always use traditional keyword search systems and use these AI search as a supplement method.

The tricky bit is the way current "semantic search" technologies work they are typically finetuned to work better or worse for different domains (so if the embedding models was finetuned with example sets from Physics it would be better at figuring out relevancy of Physics domains). But there is no way to tell in advance how the semantic search (typically dense embedding models) was trained and which domains it is good at.

Typically they tend to work better for STEM by default but the best way is for librarians to try on their own sets of queries. My review doesntreally go deeply into testing methods - of which there are many.....

Re: bioRxiv viewer

aarontay — Sun, 28 Jun 2020 23:36:51 -0000

Very nice. I was doing similar analysis for my institution and just wondering about how much the results vary when moving from WOS, Scopus, Microsoft academic as well as the range of years involved. Analysing with diff Unpaywall dumps at different time is also novel. However I wonder if the most important factor the use of unpaywall itself might be worth studying using other services like CORE discovery or Open access button APIs.

Re: 4 different ways of measuring library eresource usage

aarontay — Sun, 12 Mar 2017 09:36:25 -0000

Thanks. But I wouldn't hold my breath. Past history has shown very few people comment though I know many find the content useful.

Re: Librarian & fake news - Bayes, metaknowledge &

aarontay — Fri, 24 Feb 2017 10:17:50 -0000

Thanks Lane. Edited.

enjoyed your appearance on the podcast at https://circulatingideas.co... really interesting.

Re: Library Discovery and the Open Access challenge - Take 2

aarontay — Tue, 03 Jan 2017 10:06:21 -0000

No problem.I think you are legal trained and you are trained to be very precise in language, while I'm playing futurist here and we tend to strive for ambiguity for obvious reasons. :)

I think the issue here is this. Firstly, I write very broadly and "academic libraries" can range quite a bit, though I try to address the majority covered by this term. Secondly, I have written on this topic over 2-3 years from various angles it gets tiresome to state all my assumptions and arguments over and over again and it's easier to write broadly and briefly without qualifying over and over.

My interest in this topic began in 2011 where I started writing about "Web scale discovery services", the so called "one search" and a lot of what I mean by discovery can be understood from that angle, though if you say discovery is more than that I agree fully.

The next leg of the argument relies in believing that open access has a momentum that is unstoppable and it's a matter of when not if open access will be dominant.

Combine the two and you have my question.

To your questions

1. I do agree nothing will and can be 100% free. My argument currently applies more towards the STEM and to some extent the social science disciplines and more particularly on journal articles and maybe books (if Open access book models actually catch on). Even within this domain, does that mean "everything (to be fair I qualified with almost)"I also seriously doubt expensive finance databases datasets from Thompson reuters bloomberg etc will become free even in the next 20 years. I'm sure you can think of other examples.

One can also quibble about the value of curated data for discovery, on the value of A&Is (abstract & indexing) databases over just full text etc. But I think rather than becoming hung up on the word "Everything" (again notice I qualified with almost at the very beginning) consider this. It's 1990 and someone told you that in 25 years' time, "everything" would be on the web and libraries would no longer be the sole or even dominant source of information for most people. General reference questions would dry up and your reference books and encylopedias would fall into disuse compared to Wikipedia. Someone in 1990 could correctly state this won't happen as not everything will be online and be 100% technically correct that not "everything" is online and there are some niches where library resources are needed beyond searching Google....

2. Hmm law. Specialised fields like law, music are generally worlds in themselves so what I write generally doesn't apply to them. But yes without quibbling too much discovery here means research or even more specifically since I'm talking about library search, searching using keywords.

3. No technical meaning of "solution" was meant here. Replace with service if you like

Re: Library Discovery and the Open Access challenge - Take 2

aarontay — Tue, 03 Jan 2017 09:30:29 -0000

Thanks Leslie for your support! I will try my best. Happy 2017!

Re: 5 reasons why library analytics is on the rise

aarontay — Mon, 21 Nov 2016 21:09:23 -0000

Hi, it's just a google sheets created by myself. I've fixed the link.

Re: 5 reasons why library analytics is on the rise

aarontay — Mon, 21 Nov 2016 06:46:27 -0000

Thanks for stopping by Stephen.

The top 50 board idea reminds me of the gamification system lemontree that universities like Huddersfield has implemented.

https://library.hud.ac.uk/l...
http://librarygame.co.uk

Re: Are institutional repositories failing?

aarontay — Thu, 18 Aug 2016 23:08:10 -0000

Thanks for stopping by with the comment Richard.

Re: Are institutional repositories failing?

aarontay — Thu, 18 Aug 2016 23:07:32 -0000

You can easily get a list of institutional repositories by looking at http://roar.eprints.org/ . There should be papers like http://www.dlib.org/dlib/ja... .

The tough part as always is measuring value.

Now Universities are looking into CRIS systems like Pure and Converis , the cost of those probably dwarf that of IRs but then again the main functional of such systems is not to host papers open access so might not be comparable.

Re: Are institutional repositories failing?

aarontay — Thu, 18 Aug 2016 22:52:29 -0000

Oops typo. Thanks updated.

Re: So Hey You Should Stop Using Texts for Two-Factor Authentication

aarontay — Sun, 26 Jun 2016 14:13:31 -0000

Most systems that provide 2FA even if they allow Google Authenticator support will almost always allow SMS as a backup. So they don't really help, since a hacker who wants to exploit SMSs can still choose to switch to SMS as the second factor.

Re: Ezpaarse - a easier way to analysis ezproxy logs?

aarontay — Tue, 22 Mar 2016 11:00:34 -0000

Yes. Because the ezproxy logs capture every http request this means even in one session there will be several lines for image files, CSS files etc. Even for small institutions, you can easily get over the limit for rows for fairly short periods.

Re: Libraries and Trello - How are librarians using it?

aarontay — Tue, 22 Mar 2016 06:05:01 -0000

Stephane, when I was preparing this blog post on Trello use last year, I did in fact come across your Trello board on ezpaarse, but at the time I had no clue what it was about :). Now of course I do as blogged in my latest post on ezpaarse....

Re: Primo and Summon - Same but different? - (I)

aarontay — Thu, 10 Mar 2016 10:20:58 -0000

I really have no insight on how it works.

I can wildly speculate that Discovery vendors can try to impose standards but if the content providers refuse to do so or don't have the ability to, it would mean either a) totally rejecting the content and/or delaying indexing b) accepting the content which is better than nothing

That said I was at a conference once and I heard a content provider complaining that the discovery vendor didn't use the quality metadata they provided properly in their indexes......

Re: Primo and Summon - Same but different? - (I)

aarontay — Fri, 04 Mar 2016 05:37:50 -0000

Hi , yes I know what you mean, you do a search and the first page say 8 out of 10 are the same title that should have been merged. But sadly that is the nature of Web Scale discovery these days.

Discovery vendors can't do much if the underlying metadata is not clean nor complete despite how much "normalization' they try to do.

I speculate that discovery services that tend to heavily weight title fields will tend to throw up "pages of duplicate titles", and so a partial workaround for this issue is for the relevancy algorithm to try to throw some variety into the first page of results so it is unlikely to list duplicates together.

But this isn't a complete solution, because a failure to match and merge for print items can have consequences.

Interestingly, I think how and what Primo and Summon's does grouping/match and merge on is also slightly different .....

Re: Measuring the value of contributed metadata in web scale discovery services

aarontay — Thu, 17 Dec 2015 10:12:54 -0000

Yes "Google" string in source here means link resolver requests clicks in Google Scholar search results.

It's an exception to the general rule of needing colons in the string to represent openurl requests from external databases.

Re: [Research question] What percentage of citations made by our researchers is to freely available content?

aarontay — Sun, 20 Sep 2015 23:53:40 -0000

Hi karen

Thanks for the valuable comments, yes the sampling are drawn from different populations so in theory they shouldn't compare, but they are the closest comparisons I could get to see if I am totally off base.

Such studies usually but not always show open access articles gets more cites, I am trying to flip it over on the head and ask the other way, of the papers that were cited, how many were free.

It occurs to me that studies that try to tease out correlations between "free" (beyond OA) and cites can often run into the same issues with timing. A simple minded correlation between free now and high cites likes the ones I cited above runs into the same issues if one cares about causation. Do people cite "Free" because they can access it easily to cite? If so shouldn't one take into account when the item is free? An item published 5 years ago in a IR with 2000 cites may strengthen the correlation but if you find out it will just put in the IR yesterday then it can't possibly affect the citations at least not in that way.

Of course studies that focus only on Gold immediate OA and cites should not have this issue because they are most always free at time of release. Equivalently an easier study for me would be to ask, of the citations made by faculty, how many were to Gold Journals (at the time). But I suspect the results would be very low.

I think my research question is obviously very important, since it has implications on some fundamental open access issues but I am not sure if I can get at it. Thanks for the reminder to consider other non-bibliometric techniques. Will continue to think of other ideas as well.

Thanks again.

Re: 5 things google scholar does better than your library discovery service

aarontay — Sat, 18 Jul 2015 13:17:58 -0000

Thanks François for the interesting comment.

Hopefully all the discovery service providers will take note of this weakness with OA materials and work on improving it.

Re: How academic libraries may change when Open Access becomes the norm

aarontay — Thu, 21 Aug 2014 11:21:07 -0000

"Library heading towards a utility role" indeed. You may be right, but I think this point of view is not so easy to see and many would disagree.

But the triumph of Open Access, literally forces libraries and hence librarians out of the role that you don't feel has much of a future.

One librarian told me, after reading my blog post, she was literally worried for the future of librarians. I said, my post did not predict the end of librarians and suggested where we could and should go but she mentioned that it was weird to have librarians who didn't act like librarians.

So the role of libraries as buyers has a very strong hold on the minds of some.

Re: How academic libraries may change when Open Access becomes the norm

aarontay — Thu, 21 Aug 2014 11:10:16 -0000

Sheila, abstract and indexing (A&I) databases is something I was considering mentioning but the blog post was long enough but you have a good point.

Still, as I understand it, right now the rise of Google Scholar and web scale discovery has led many libraries to consider if A&I databases are worth the price, or would users prefer they spend money on full text (which is getting more expensive).

As librarians you and I know that A&I provides more precise searching, has unique features, and covers print and other grey literature that might be hard to find. The question is how much do our users value it? Or do they think full-text searching + algorithmic features (that don't involve paying for painstaking human expertise to create eg indexing) to do relevance ranking is almost as good as indexing + controlled vocabulary.

Ideally, the story should go, the world shifts to open access via a transition that results in cost savings. As Academic libraries no longer need to pay for full text packages this frees up the budget to spend on A&I databases.

Meanwhile publishers/library vendors seeing the winds are shifting start to devote resources to package free full text with innovative features that are specialized for the needs of a specific audience and provide a compelling reason for use that no general tool can even come close to matching.

I have no idea if this will happen. The cynic in me suspects that while the big names like Web of Science, Scopus , Psycinfo, Pubmed will survive, the majority won't. The former two especially would survive due to the citation data that is highly influential determining promotion and tenure decisions (though Google Scholar might disrupt this). The latter two have audience mass and a near monopoly as the universal go to tool for their domains and also have very specific, unique features, general search can't match.

But my impression of most A&I are that besides the big few, resources aren't been pumped into improving the interface and features, and they are often stuck with old outdated interfaces. Perhaps everyone is concentrating on trying to create open access journals?

So if you ask me, sure academic libraries won't lose all their budget, they may not need to pay for full text journal articles platforms, they will still need budgets for other types of databases, newspapers databases, financial/Statistical data etc.

Will they have budgets to pay for A&I databases? It all depends perhaps on how the OA transition occurs.

Let me paint a worst case scenario.

The rise of open access, means more and more users feel they don't need to use library sources including A&I databases. Usage of A&Is which had already declined due to web scale discovery* and google scholar plunge to almost nothing.

In this very negative scenario, Academic libraries or the institutions they are part of do not get any savings because subscriptions get captured in APCs. During the transition, under pressure, academic libraries cut A&I budgets to fund this transition - perhaps under a double dipping scenario where they pay for APCs while not daring to cancel subscriptions........

On a more positive scenario, academic libraries "Take back" the Scholarly communication system. And innovate along the roles you mention, perhaps improving how layered journals work etc.

* Due to disagreements on how to credit A&I source results in web scale discovery results, A&I refused to be included.

Re: The Library Isn't Flat

aarontay — Sat, 05 Jul 2014 12:31:42 -0000

Ken Varnum at the latest Top Tech Trends talked about
Personally-tuned discovery system.

https://www.youtube.com/wat...

He makes a interesting point, what's the difference between Google Scholar and your library's discovery service since from the point of users they are both big oceans of papers?

His solution which also seems to be what you are talking about, is where "librarians with a little bit of work" can create subsets of results both at the subject level and the appropriateness of the way people at your institution do things.

Is sounds great, though I am a bit doubtful about the "little bit of work" part. The 2 commercial discovery systems I know of Summon and EDS do have hardcoded "disciplines" and some such as Primo Central even allow you to login and select your discipline and level of knowledge but in general I think they don't work very well perhaps because they are still too broad (based typically on simple mapping from call number, the journal they are in, and/or subject terms).

I am not sure how easy it would be local librarians to "slice up" the collection effectively.

Simplest could be like the days of federated search where we chose particular collections/databases (with current technology we could even do journal level) for each subject but would that be granular enough?

Have no idea how it would work for other ways such as appropriateness of material.

"Is there a way to help people who want to join those conversations see the patterns and discern which ideas were groundbreaking and significant and which are simply filling in the details? "

The closest to “trails of association" I can think of in the academic library conext is BX recommender which looks at which articles tend to be viewed together and uses that information to associate and create recommendations is http://www.exlibrisgroup.co...

To go beyond that you would have to get people to indicate what they found really useful (how do we get people to do that?). There is some talk I think also about coding the type of citations received.

Re: Types of librarian expertise - are they getting easier to acquire?

aarontay — Thu, 12 Jun 2014 11:04:12 -0000

I was thinking of expertise not so much the content we provide. But you are of course right, particularly with Open Access.

If you ask me, we are losing in 2 areas

1) They don't come to us for the content

2) They don't come to us FIRST to search/discover.

Re: Types of librarian expertise - are they getting easier to acquire?

aarontay — Thu, 12 Jun 2014 11:00:24 -0000

I am always fascinated by users who are self-taught on use of library services. They may know some things but have big gaps in others - the don't know what I don't know problem. For some users, who like to learn on their own pace, often I find all they want to know is that something can be done and they are confident they can figure it out (and are right).

So I guess some expertise might be easier to acquire but only if you knew it existed.

Also I was thinking more in terms of expertise rather than outright providing resources. My concern is if we are pushed towards a situation where our pocketbooks are no longer what we exist for, coupled with expertise been easier to acquire, what do we exist for?