Organizing your Systematic Review: Project Notes Template for Librarians

Updated January 15, 2017

Do you ever find that sometimes it’s the most practical information that we often fail to share or teach?

As a librarian designing search strategies for systematic reviews and supporting review teams for over two years now basically as a full-time job, I feel like I’ve finally streamlined things. The processes that I’ve developed make planning, executing, and reporting the search strategy for a systematic review really straightforward – but it wasn’t always this way. Everything I use is based on trial and error, and in speaking with other librarians. 

Knowing that this kind of uber practical stuff isn’t often shared online, I figured I’d put some of it up in the hopes that maybe some of you might find it useful, or better yet, generate discussion and maybe sharing of your own tools or processes you use to keep things organized (email meeee!).

For starters I figured I’d begin with the core working document that I use basically from day one, and which absolutely saves my butt every time we roll around to manuscript writing time, because it contains every conceivable detail and thought process throughout the search strategy design:

Systematic Review Project Notes Template for Librarians

http://bit.ly/Template-SRProjectNotes-SV 


Systematic Review Project Notes Document by Sarah Visintini is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License

Creative Commons License

The Way I Use this Form:

Team & Project Information

Basically how I use this thing is I keep all the research group and project information at the top, along with any key decision items, so that: a) I don’t have to go digging through emails to see what we ended up deciding on or b) so that I’m not constantly asking them what databases they have considered or review management software they want to use.

When I’m juggling lots of systematic reviews at the same time keeping them all straight from memory alone can be challenging, and I rely on this section of the document to help me re-immerse myself in the important details of the project, either when I’m ready to have another crack at the search strategy design, or in preparation for a meeting with the investigators.

Databases Table

This table remains empty for the majority of the time that I’m working on this document, but I like to keep it at the top so that when I come back to these notes to write the search methods section, most of my key information is front and centre. I also like to copy and paste this table into my final search results email with researchers so that they have a lot of the core information regarding what databases were run and when for their own records.

Search Strategy Design

This is probably the messiest section and has most of my testing and ideas popped in. I don’t usually use this section much after the fact (favouring the search histories and databases table for the majority of my write up) but it doesn’t hurt either.

Relevant Systematic Reviews

This is a new section I have recently started adding to prompt me to make sure I search for any reviews on the same topic in case the topic has been scooped or there is overlap, as well as to have a tidy place to list out some of the search strategies others have used for similar (but not significantly overlapping) topics. So for example if I was conducting a systematic review on X and Y I would have two subcategories here where I’d copy and paste some good search strategies for X and some good ones for Y so that I could make a beautiful Frankenbaby search* and ensure I wasn’t missing any good terms. 

*This may also come into play come manuscript writing time, when you might be citing the review as the basis for your own search strategy.

Target Articles

Any articles that the team provide to me as being “target articles”/”pearls”/”definite includes” go in this section. Often I will have them in a variety of formats (in a reference list, in a PubMed PMID list for search testing, or in the Ovid Medline .ui. chain – again for search testing). If I end up finding some additional articles that the team deems include-worthy I will also add these here, usually with most current at the top, so that I always know that I am using the latest list of target articles.

Emails

I obviously don’t include everything here, but if there were any decision points in the emails I exchange between myself and the researchers I like to paste them in here for reference so that I can just skim the essentials rather than digging through email threads. This has actually saved me  when I went back to write search methods for a review after having moved to a new position – I had lost all my old emails but still had the salient details in my project notes.

Search Histories

Arguably the most important part of the document, I copy and paste my search history every time I search a database for the project. Typically by the end of a search strategy design process there will be between 5 and 15 entries here by date and database, so that if I have any doubts or questions I can go back through my process and look at what my searches were pulling in. On particularly tricky searches or when I’m testing out a lot of stuff, I’ll often leave myself notes using the Comments features in Word. This has actually come in handy a number of times when either peer reviewers or researchers have asked, “Have you thought of using term….?” and saves me time duplicating tests I’ve already conducted.

Happy systematic searching!

MLA CE: Advanced Searching Techniques and Advanced Strategy Design

Wow, what a rush! It’s already been a month since the MLA/CHLA/ABSC/ICLC Mosaic Conference wrapped up in Toronto and my head is still buzzing.

Besides getting to meet a whole host of incredibly cool and interesting medical librarians (more on that later, if I can muster the blogging muscle), I also had the privilege to attend one of the pre-conference continuing education sessions “Advanced Searching Techniques and Advanced Strategy Design” led by Julie Glanville and Carol Lefebvre.

If those names sound familiar to you, it’s because they are involved in all sorts of really interesting work in the systematic searching world. You may have seen their names in the Cochrane Handbook, on the ISSG Search Filters Resource website, or in CADTH’s PRESS Peer Review of Electronic Search Strategies: 2015 Guideline Explanation and Elaboration. It was kind of funny really, I had recognized their names on the CE materials, and frequently use ALL of these resources, but didn’t put it all together until the session itself (needless to say, I was pretty star struck when all the puzzle pieces came together).

The session was broken up into four main components: search term identification, identifying, choosing and using search filters, search strategy structure, and peer review of complex search strategies.

Search Term Identification

During this part of the session, we discussed what techniques we used to get a search rolling, and Glanville walked us through a number of really interesting resources, including the Yale MeSH Analyzer, PubReMiner, GoPubMed, MeSH on Demand, and Quetzal.

Text analysis tools for the win!

My key takeaway from this part of the course: using text analysis tools can really enhance the search at the creation stage.

Text analysis tools are fast and can give you an instant breakdown of how frequently keywords and MeSH headings are used in a set of target articles. Where before I would have gotten started on a search by identifying a number of target articles and manually scanning them for commonalities (and more recently using the Yale MeSH Analyzer to identify common MeSH terms), using tools like PubReMiner makes this process faster, and also leads to the use of terms and headings that one may not otherwise have picked up on, which will (hopefully) enhance the precision of searches.

Search Filters

In this session, Lefebvre described the processes in which search filters have been designed, gave a brief background on the ISSG Search Filters Resource, and took the group through methods for appraising search filters.

Search filters: to cite or not to cite?

We got into a really interesting discussion as a group when one of the attendees asked our facilitators whether librarians should be citing the use of search filters when they write up their methods.

Lefebvre was quite adamant that all search filters should be cited, published or not, modified or not. She asserted that if modified, the filter should still be cited, but a description of modifications should also be included.

For her part, Glanville felt that modified search filters were essentially new creations and it might be misleading to cite a validated filter (since modification would render validation invalid).

This was a really important conversation to have as a group since I will (somewhat abashedly) admit that though I take painstaking notes of whose searches helped inform my own, it doesn’t always occur to me to cite them when I write up my search methods. Lesson learned!

Search Strategy Structure

What do you do when a search question doesn’t fit into a tidy PICO breakdown? During this session of the day, Glanville walked the group through other search mnemonics such as SPICE (qualitative), ECLIPSE (management) and variations on PICO such as PICOT-D. We also spent some time as a group developing searches that are broken down into concepts, but which don’t fit into tidy search mnemonics. As Glanville pointed out, this strategy can be useful when searching topics in public health, epidemiology, adverse effects and quality of life.

Peer Review of Complex Search Strategies

During this final session of the day, Lefebvre discussed search strategy peer review with the group, and took us through the Peer Review of Complex Search Strategies or PRESS project and its 2015 update. We also spent some time individually assessing a search strategy with errors embedded throughout and went over the search as a group to discuss various approaches and possible improvements.

Final Thoughts

All in all, this was a very useful course, and I have already attempted incorporating text analysis tools in my search strategy formulation (saving time and identifying search terms that wouldn’t normally have been on my radar in the process).

I would say that our session was excellent for two main reasons:

  • Julie Glanville and Carol Lefebvre are incredibly knowledgeable and experienced. They are a veritable wealth of information and are both searching powerhouses.
  • Thanks to our small class size and the way in which Glanville and Lefebvre organized the session, we also got to learn from our colleagues quite a bit in small and large group discussions – you wouldn’t believe the collective brain power we had in the room that day!

Many thanks to the Glanville and Lefebvre for their enthusiasm and knowledge, to my fellow attendees for sharing their own excellent ideas and thoughts with the group, and to my workplace, the Maritime SPOR SUPPORT Unit for sponsoring my attendance at this session!