HMSOM Medical Elective: Developing Search Skills

What is Grey Literature

The term grey literature is used to describe a wide range of different information that is produced outside of traditional publishing and distribution channels, and which is often not well represented in indexing databases.

A widely accepted definition in the scholarly community for grey literature is

"information produced on all levels of government, academia, business and industry in electronic and print formats not controlled by commercial publishing" ie. where publishing is not the primary activity of the producing body." 

From: Third International Conference on Grey Literature in 1997 (ICGL Luxembourg definition, 1997  - Expanded in New York, 2004).

When conducting a systematic review it is important to include literature that has not formally been published in sources as it helps to prevent publication bias. An exhaustive literature search, especially one that leads to a systematic review, needs to include searches in gray literature, to overcome, or at least minimize, “publication bias” or “reporting bias”. Searching grey literature is supported/mandated by the Cochrane Collaboration, the Campbell Collaboration, JBI and the Institute of Medicine (U.S.)

Pros & Cons of Grey Literature: Uniqueness and Currency VS. Quality and Longevity

Grey Literature can be a very important research resource.

• It can record findings in niche or emerging research areas and also record research findings that produce null or negative results.  These may not be addressed by the commercial publishers - who may have a more mainstream, profit based publishing strategy.
• It may be more current than formally published research literature which can take time to pass through a sometimes lengthy peer review and editorial publishing process.
• It can connect you with content from a more diverse range of authors and institutions.  Not everyone is able to publish through commercial publication routes

You should be aware of the need to assess and capture potentially useful resources

• Grey Literature sources can vary hugely in terms of quality.
• Scholarly publications such as books and journals pass through a formal publishing process and are subject to in depth editing review. In many cases there is a peer review process where draft versions are subject to a scrutiny process by a panel of experts.  Papers may be modified and improved in light of the expert feedback before reaching final published status. 
• Not all grey literature material is subject to a similarly rigorous pre publication review process, so you should exercise caution and make your own assessment for quality, reliability and potential bias.  
• Grey literature is often not formally published. For example, a report may be posted to a government website or a poster presentation may be given at a conference. In these cases you need to consider the longevity of the resource.  Something may be available on the web, or a blog for a short period only and may not be formally archived. 
• Make sure you keep a record of material you wish to use/reference - as it may not be there for discovery later down the line.

Examples of Grey Literature

A wide, and growing, range of material can be considered as grey literature. For example, clinical trial information is primarily of interest to health and medical research.  You should consider which types of information you are interested in before you begin searching as this will help you target and frame your online searching.

Examples include:

This is only a sample of Grey Literature sites

• Grey Matters: A practical search tool for evidence-based medicine
• Kennedy School of Government Think Tank Search, a custom Google search of over 670 think tank and research center websites  https://guides.library.harvard.edu/hks/think_tank_search
• MedNar:  deep web search engine of 60+ authoritative medical collections  www.mednar.com/mednar

Foundations

• Robert Wood Johnson Foundation - RWJF has worked to improve health and healthcare in the United States. Health systems, health disparities, social determinants of health, etc.  https://www.rwjf.org/
• Kaiser Family Foundation KFF (Kaiser Family Foundation) is a nonprofit organization focusing on national health issues, as well as the U.S. role in global health policy. KFF develops and runs its own policy analysis, journalism, and communications programs, sometimes in partnership with major news organizations.   https://www.kff.org/

Reports and Research

• NIH RePORTER: Describes research projects funded by NIH and other DHHS agencies.  https://reporter.nih.gov/
• Virtual Technical Reports Center: List of institutions with links to technical reports, preprints, eprints, reprints, dissertations, theses, and research reports of all kinds  https://lib.guides.umd.edu/techreports

Clinical Trials

• CenterWatch: Global source for clinical trials information, offering news, analysis, study grants, career opportunities, and trial listings to professionals and patients  www.centerwatch.com
• ClinicalTrials.gov: Registry and results database of federally and privately supported clinical trials conducted in the United States and around the world   https://clinicaltrials.gov
• EU Clinical Trials Register: Registered clinical trials in the European Union   https://www.clinicaltrialsregister.eu/
• ISRCTN Register: Registered randomized controlled trials and other studies to assess the efficacy of health care interventions around the world   https://www.isrctn.com/

Health Statistics

• National Center for Health Statistics

Wide variety of health-related statistics, including FastStats for quick summary data.

• Data.gov

Descriptions of Federal datasets (metadata), information about how to access the datasets, and tools that leverage government datasets.

• Data.gov: Health

Datasets, tools & applications using data about health and health care gathered from US Federal government agencies. Social-science data and statistics from a variety of sources.

• American Factfinder - U.S. Census data

Interface for data from Census 2010 and 2000 and the 2010 American Community Survey. Tables and maps for U.S states, counties, cities, towns, by race, ethnic, ancestry, or tribal groups.

• Partners in Information Access for the Public Health Workforce

Health data tools and statistics.

US Vital Statistics/ Health & Disease

• Atlas of United States Mortality - In addition to maps with age-adjusted death rates for each HSA, the atlas includes maps that compare each HSA rate to the national rate, smoothed maps for each cause that show the broad geographic patterns at selected ages, and a chart with regional rates for each cause of death.
• Bureau of Labor Statistics: Injuries, Illnesses, and Fatalities - Annual information on the rate and number of work related injuries, illnesses, and fatal injuries, and how these statistics vary by incident, industry, geography, occupation, and other characteristics.
• National Institute for Occupational Safety and Health (NIOSH)- Data and statistics from NIOSH projects that research and monitor worker health and safety. Includes the the National Occupational Mortality Surveillance (NOMS) database, which monitors patterns of cause of death (including diseases) by occupation and industry.
• CDC Wonder Searchable compilation of Centers for Disease Control (CDC) reports, guidelines, and health data for disease and demographic groups. Includes surveillance data, Healthy People 2010, MMWR, and other CDC publications.
• Healthy People DATA2020 Interactive data tool to explore data and technical information related to the Healthy People 2020 objectives.
• MMWR Weekly data for notifiable diseases in the U.S. Includes annual summaries, guidelines, and trends.
• SMART: BRFSS City and County Data Data of selected metropolitan and micropolitan statistical areas (MMSAs) from CDC Behavioral Risk Factor Surveillance System (BRFSS)

Women’s Health

• FastStats: Women's Health- CDC NCHS
• MCH Digital Library- National Center for Education in Maternal and Child Health, Georgetown University.
• State Profiles for Women's Health -Kaiser Family Foundation. Interactive U.S. map and dashboard.

Youth and Aging in the US

• AgingStats.gov- Federal interagency forum on aging-related statistics
• Alzheimer's Disease and Healthy Aging Data Portal - CDC. Explore data by State.
• Kids Count Data Center - Measures of child well-being, including education, economics, family and community, health, safety and risky behaviors. View community-level or state data, or compare cities and states across the U.S.
• State of Aging in America: Data Briefs - Topic-specific documents focusing on public health issues related to older adults.
• Youth Risk Behavior Surveillance System (YRBSS) - National and state surveys monitoring priority health-risk behaviors and the prevalence of obesity and asthma among youth and young adults.

US Health Services

• AHRQ: Data Resources Databases
• County Health Rankings - Collaboration between RWJF and U of Wisconsin Population Health Institute
• HealthData.gov - Access to U.S. Health & Human Services data. Includes topics of health care workers, health care services, social and family services, finance and expenditures, income and poverty.
• Kaiser State Health Facts - State-level data on demographics, health, and health policy, including health coverage, access, financing and state legislation.
• HCUP - State, regional and national level discharge data, available for purchase. Includes State Inpatient Databases (SID), State Ambulatory Surgery Databases (SASD), State Emergency Department Databases (SEDD), Nationwide Inpatient Sample (NIS), and Kids' Inpatient Database (KID). Free tools include AHRQ Quality Indicators (QIs), Clinical Classifications Software (CCS), Comorbidity Software, and HCUPnet, a web interface to free HCUP data.
• HSR Information Central (HSRIC) - National Library of Medicine portal of health services research resources including links to data, funding announcements, reports, podcasts, & discussion groups.
• HSRR (Health Services and Sciences Research Resources) - Data resources used in health services research, the behavioral and social sciences and public health.
• Dartmouth Atlas Project - Uses Medicare and Medicaid data to provide information and analysis about national, regional, and local markets, as well as hospitals and their affiliated physicians

International Statistics

• World Health Organization: Global Health Observatory Data Repository - Guide to health and health-related epidemiological and statistical information available from World Health Organization (WHO). (Formerly WHOSIS: WHO Statistical Information System).
• Global Health Data Exchange (GHDx) - IHME's (Institute for Health Metrics and Evaluation) catalog of the world's health and demographic data. Use the GHDx to research population census data, surveys, registries, indicators and estimates, administrative health data, and financial data related to health.
• CIA World Factbook - Maps, general demographic and statistical information by country. Includes economics, life expectancy, fertility and death rates.
• WHO Global Atlas of the Health Workforce - Includes statistics on the numbers and density of health workers for all 193 Member States.
• WHO Mortality Database - World Health Organization (WHO) database. Provides annual reported data on mortality statistics by age, sex and cause of death.
• World Health Report (WHO) - Each year the report combines an assessment of global health, with a focus on a specific topic such as maternal & child health, health promotion, health system improvement, health systems financing, etc.
• World Health Report Statistical Annex - Links to graphs and tables of past World Health Reports (WHO). Collected data for most of the world on morbidity and mortality, by age, sex, and cause.