Data collection and filtering

Our study workflow is outlined in Figure 1 . Publicly available tweets, messages posted on Twitter, were collected through the Python library GetOldTweets3, which harvests within a specified time frame tweets with any combination of specified words or characters. We searched among English-language tweets posted between March 21, 2006, and December 1, 2019, using a list of 112 key words ( Supplemental Table 1 ), including brand and generic names, common colloquial names, and abbreviations of the abovementioned contraceptive methods. We created 8 categories of tweets mentioning the following 6 reversible contraceptive methods: copper IUD, levonorgestrel IUD, IUD-type unspecified, implant, OC pills, contraceptive patch, vaginal ring, and Depo-Provera shot.

Study workflow

General workflow of data extraction, processing, and analysis. Tweets mentioning any of 6 contraceptive methods (in 8 classes, including copper IUD, levonorgestrel IUD, IUD-type unspecified, implant, oral contraceptive pills, contraceptive patch, vaginal ring, and Depo-Provera shot) were harvested. Sentiment analysis was conducted to answer the question, “Is each tweet mentioning a single contraceptive method positive, negative, neutral, or mixed emotion?” The sensitivity and specificity for the detection of positive, negative and neutral sentiments toward the contraceptive method were calculated on the basis of the manually curated tweet collection.

IUD , intrauterine device; NLP , natural language processing.

Merz et al. Attitudes toward contraception on Twitter. Am J Obstet Gynecol 2021 .

We removed tweets for which the tweeter’s username contained a search key word (eg, @NuvaRingLawyer) and tweets related to male or emergency contraception (containing the terms “male contraception,” “male contraceptive,” “male birth control,” “emergency contraception,” “emergency contraceptive,” and “emergency birth control”).

To characterize trends in the number of tweets mentioning each contraceptive method over time, we counted the number of tweets per year per contraceptive method category. To assess for statistically significant trends over time, we used Mann-Kendall nonparametric tests for monotonic (ie, constantly increasing or decreasing) trends in the number of tweets about each method over time. All testing were 2-sided. A Bonferroni-corrected P value of <.006 was considered statistically significant.

Sentiment analysis and manual curation

In NLP programs, computers automatically process and analyze text. NLP sentiment analysis enables rapid, large-scale identification of opinions expressed within texts, most often categorizing text as having positive, negative, or neutral sentiment. For example, a sentiment analysis program would automatically categorize the sentence “I love my Mirena IUD” as positive ( Table 1 ). We performed sentiment analysis on tweets mentioning only 1 contraceptive method. Specifically, we used the Amazon Web Service (AWS) NLP program, Amazon Comprehend, which classifies text as positive, negative, neutral, or mixed sentiment, with an associated confidence score ranging from 0 to 1; a lower confidence score indicates concern for lower accuracy of sentiment detection.

To assess NLP sentiment analysis accuracy, we compared the Amazon Comprehend output to that of a manual sentiment analysis performed by 10 independent reviewers (A.A.M., M.H., A.M., C.Y.L., A.O., R.S.S., K.S., A.S., N.W., and S.Y.). We randomly selected 1000 tweets and divided them into 2 groups of 500. Reviewers were divided into 2 groups of 5, and each reviewer independently and in blinded fashion categorized the emotional tone (positive, negative, neutral, or mixed) of each 500 tweets with respect to the contraceptive method it mentioned. A tweet was categorized as “false positive” if it did not mention a contraceptive method. If at least 3 of 5 reviewers agreed on a tweet’s sentiment, we considered it a true sentiment of the tweeter with regard to the contraceptive method referenced. For example, 4 of 5 reviewers might assign the sentiment “positive” to the tweet “appreciating my Nexplanon today.” Tweets lacking agreement were discarded from the final analysis. For example, if the tweet “birth control pill got me feelin feels” was categorized as “mixed” by 2 reviewers, “neutral” by 2, and “positive” by 1, the tweet would be discarded. We calculated the interrater reliability (Fleiss’ kappa score) and percentage of the majority consensus (“observed agreement”) between reviewers. The final collection of manually reviewed tweets and assigned sentiments is publicly available at: https://github.com/hms-dbmi/contraceptionOnTwitter/blob/master/finalGoldStandard.csv .

To determine which NLP confidence level was most accurate, we derived the sensitivity and specificity of the automatic sentiment analysis at different confidence levels (all levels, ≥80%, ≥90%, and ≥95%) compared with the sentiments assigned by most manual reviewers.

To characterize how sentiments of tweets mentioning only 1 contraceptive method changed over time, we counted the number of tweets per contraceptive method category and sentiment category (positive, neutral, and negative) per year. Tweets categorized as “mixed” sentiment were excluded from analysis because of the low sensitivity of the automatic sentiment analysis (0% sensitive) based on manual curation. To assess the temporal trends in tweet sentiment, we used Mann-Kendall nonparametric tests for monotonic trends in the number of positive, neutral, and negative tweets over time. To compare proportions of positive, negative, and neutral tweets about a given method or method class, we used chi-square tests. All testing were 2-sided. A Bonferroni-corrected P value of <.002 was considered statistically significant.

This study was conducted using Python 3 (Scotts Valley, CA) and R (version 3.6.1; R Foundation for Statistical Computing, Vienna, Austria). Open-source code is publicly available on GitHub: https://github.com/hms-dbmi/contraceptionOnTwitter . This study was deemed institutional review board (IRB) exempt by the Harvard Medical School IRB based on its being nonhuman subject research.

Data collection, filtering, and characterization

We collected a total of 989,627 tweets that referenced at least 1 contraceptive method. We excluded 150,888 tweets that referred to male or emergency contraception or bore 1 of the contraceptive key words in the Twitter username ( Figure 1 ). To characterize the sample of 838,739 remaining tweets, we calculated the percentage mentioning each method during the study period. LARC methods were named more than SARC methods (58% vs 42%). Within tweets mentioning LARC, the most commonly referenced method was the IUD (73% of LARC-related tweets). Within tweets mentioning SARC, the most commonly referenced method was the Depo-Provera injection (42% of SARC-related tweets) ( Figure 2 , A).

Number of tweets mentioning each class of contraception since 2006

Number of filtered tweets mentioning each class of contraception since 2006 in aggregate ( A ), per year ( B ), and adjusted for the total number of mentions of contraception per year ( C ). Black bars in ( B ) and ( C ) represent the division between SARC (below) and LARC (above). Tweets mentioning multiple contraceptive methods were scored as belonging to each method category once, meaning that a single tweet could contribute to multiple categories.

IUD , intrauterine device; LARC , long-acting reversible contraception; LNG , levonorgestrel; SARC , short-acting reversible contraception.

Merz et al. Attitudes toward contraception on Twitter. Am J Obstet Gynecol 2021 .

We then calculated the number of tweets mentioning each contraceptive method per year. We found that the number of tweets per year that specified any LARC method and contraceptive pills increased over time (all P <.006). In contrast, there was no monotonic upward or downward trend in the number of tweets about hormonal patches, rings, or injections over time (all P >.006) ( Figure 2 , B). The proportion of tweets that mentioned LARC generally increased over time, with the smallest proportion posted in 2009 (22.4%) and the largest in 2019 (76.3%) ( Figure 2 , C).

Sentiment analysis

Of the 838,739 tweets mentioning at least 1 contraceptive method, only those that specified a single method (n=665,064) were included in sentiment analysis. We further narrowed the remaining data set to tweets that earned a sentiment analysis confidence score of ≥0.95 (n=160,713). This cut-off was determined by assessing optimal sensitivity and specificity of NLP when compared with manually-curated tweet sentiment with respect to the contraceptive method mentioned. Using manual curation of sentiment, we identified 889 tweets (91.4% of total analyzed) that achieved majority agreement by reviewers (interrater reliability Fleiss’ kappa scores were 0.632 and 0.534 for the 2 groups). Based on manual curation, NLP sentiment analysis with a confidence score of ≥0.95 was 67% sensitive and 92% specific for positive, 83% sensitive and 71% specific for negative, and 74% sensitive and 88% specific for neutral sentiments ( Supplemental Table 2 ).

Of the 160,713 tweets with a sentiment confidence score of ≥0.95, the greatest proportion overall (40.66%) was negative ( Table 2 ). The collection referring to the Depo-Provera shot had the greatest proportion of negative tweets (61.49%), whereas the group referring to hormonal IUDs had the smallest (15.82%). By contrast, the greatest proportion of positive tweets occurred in those referring to the copper IUD (30.37%), whereas the smallest appeared in those referring to the hormonal patch (7.3%). More generally, significantly more positive tweets mentioned LARC than SARC (19.65% vs 10.21%; P <.002) ( Table 2 ; Figure 3 ).

Annual number and proportion of positive, negative and neutral tweets about each contraceptive class since 2006

Bars represent proportions, and lines represent numbers of tweets in each sentiment category. Mixed emotion tweets were excluded, owing to the NLP algorithm’s 0% sensitivity in detecting mixed emotions toward a contraceptive method based on our gold standard manual sentiment analysis. Mann-Kendall tests were used to test for monotonic trends in number of tweets, with the asterisk denoting P <.002. For tweets mentioning the IUD, the number of negative tweets in 2018 and 2019 were 5158 and 8654, respectively.

IUD , intrauterine device; LARC , long-acting reversible contraception; LNG , levonorgestrel; NLP , natural language processing; SARC , short-acting reversible contraception.

Merz et al. Attitudes toward contraception on Twitter. Am J Obstet Gynecol 2021 .

Temporally, we found upward trends in the annual number of both positive and negative tweets about all LARC methods (all P <.002), except for negative tweets mentioning hormonal IUD, which remained steady over time ( P >.002). There was no statistically significant trend in the number or proportion of tweets of any sentiment related to any SARC method. However, we did note small but statistically insignificant trends between 2018 and 2019 toward an increasing number and proportion of positive and negative tweets about the contraceptive pill and an increasing number and proportion of positive tweets about the contraceptive ring. The sentiment of tweets related to the Depo-Provera shot remained negative over time ( Figure 3 ).

Principal findings

The annual number of contraception-related tweets posted since 2007 has increased by nearly 300-fold. Since 2009, compared with SARC, the proportion of contraception-related tweets about LARC has increased. Using NLP sentiment analysis, we found that the number of both positive and negative tweets about all LARC methods increased significantly over time, although there was no statistically significant growth in the number of tweets of any sentiment about SARC methods. Although the greatest proportion of all tweets about contraceptive methods was negative, we did see significantly more positive tweets overall related to LARC, compared with SARC.

Results

The increasing number and proportion of both positive and negative tweets about LARC methods over time can likely be attributed to increasing use over the past decade. Previous reports have demonstrated the increasing use of LARC over our study period: for example, LARC use increased from 6% to 14% of surveyed individuals between 2008 and 2014 in the National Survey of Family Growth. Factors underlying increasing LARC use include greater availability, ^, increasing numbers of providers trained to insert LARC, high satisfaction rates, and more direct-to-consumer advertising. In 2012, LARC promotion surpassed that of contraceptive pills to render LARC the most advertised contraceptive class.

Clinical implications

The high prevalence of negative tweets overall is striking. This may be owing to a negativity bias in consumers reporting about products in general: people are more prone to report and announce feelings of dissatisfaction. Previous studies have demonstrated contraceptive users’ tendencies to emphasize negative aspects of contraceptive methods over positive ones, a phenomenon that may also be influenced by lay media historically emphasizing negative aspects of contraception.

We also observed a prominent trend toward a greater proportion of tweets with emotional valence of any kind over time. This observation may be related to the expansion of the allowed character count per tweet from 140 to 280 in November 2017, because longer tweets might have allowed for greater emotional expression. The median tweet character count in our sample increased from 77 before November 2017 to 105 thereafter, and tweets interpreted as positive and negative were longer than tweets interpreted as neutral (median character length 93 and 90 vs 70, respectively). Another factor that may have contributed to an increase in emotionally valent tweets is the phenomenon of social media functioning as “echo chambers,” in which individuals post increasingly polarized content.

Finally, the surges in annual tweets mentioning contraception, and specifically IUDs, between 2018 and 2019 are noteworthy. These increases persist even after adjusting for the estimated total annual number of English-language tweets (data not shown). These trends could be attributable—at least in part—to the growing modern feminist movement. Contraception has played a central role in the struggle for gender equity throughout history, and the IUD has become a symbol of modern feminism. In October 2017, the Trump administration expanded exemptions to the contraceptive mandate, restricting contraceptive coverage. Shortly thereafter, blocks to funding for Title X Family Planning Programs and Planned Parenthood further restricted access to family planning services. The restriction contributed to increasing political tension surrounding women’s reproductive healthcare access and the growth of a global movement for women’s empowerment and gender equity. The movement spread widely among younger generations—particularly on social media—in 2018 to 2019, with 2018 having been called “the year of the woman.”

Clinicians and public health researchers should be aware that patients may bring contraceptive knowledge, attitudes, myths, and biases formed from social media content into the examination room. With the advent of cloud computing and the increasing accessibility of large data sets containing social information, we have identified trends in contraceptive sentiment that consumers of BC are exposed to. Although this study did not assess the influence of tweets on contraceptive users’ attitudes or decision making, our findings demonstrate that many people—more than 400,000 Twitter users—have a great deal to say and may influence the public’s understanding of contraceptive methods. Recognition of this influence should lead social scientists, clinicians, and marketing companies to consider social media platforms as a potential site for consumer education. Furthermore, although our analysis did not specifically investigate misconceptions about contraception in the tweets we collected, we noted a substantial number of tweets that contained incorrect information about contraception; to this end, social media could also be used to help dispel myths and correct such misinformation.

Research implications

NLP and machine learning can augment the investigative power of traditional qualitative research, and vice versa. Our methods could help researchers and clinicians better understand patients’ needs, desires, and frustrations related to contraceptive methods, to target future research and programs more effectively. This work represents an important step toward a better understanding of the role that social media plays in contraceptive decision making, potentially opening an entirely new realm of investigation and intervention in women’s healthcare.

Importantly, this study is limited to the analysis of sentiment expressed on Twitter and does not address the influence of social media on attitudes or decision making. To assess influence, future studies could similarly employ NLP to process social media discourse through linked posts or post replies.

Strengths and limitations

This study has several notable strengths. We collected nearly 1 million tweets produced by over 400,000 Twitter users discussing contraception. We validated the performance of the NLP sentiment analysis algorithm we used with manual curation by 10 human reviewers. The human reviewers had strong observed agreement: 91.4% majority concordance. The manually reviewed collection of tweets is publicly available to train future NLP algorithms, which will enable improved sensitivity and specificity of NLP for tweet sentiment .

Although forecasting the potential impact of this work, we also acknowledge its limitations. Tweets referencing contraceptive methods with misspelled terms or uncommon abbreviations could not be collected. Tweets, including brand names without additional specifiers, were not included, because of the lack of specificity (eg, “Mirena” could refer to the IUD or a person’s name; we instead searched for “Mirena IUD”). We were not able to discern whether tweets were produced by contraceptive users, healthcare providers, institutions, or companies. Twitter users’ descriptive demographic information (eg, sex, age, ethnicity, location) is not typically posted on profiles and is not necessarily reliable when it is. For instance, location information was available for 60% of our sample ( Supplemental Table 3 ) and did not reliably refer to real locations. Many tweets express sarcasm or use emojis, images, or GIFs to convey emotion, which the sentiment analysis algorithm we used was unable to interpret. Sentiments of tweets may not be specific for the contraceptive method mentioned. Finally, it is possible that our NLP analysis overestimated the proportion of negative tweets based on the method’s greater ability to detect negative sentiment, as determined by comparison to our manually reviewed collection. This is true, even though the manually reviewed collection had a greater proportion of negative tweets than positive tweets (28% vs 20.8%).

Conclusions

Tweets about contraception, a previously unexplored and publicly available source of data, reflect contraceptive use patterns over the past 13 years and offer insight into evolving attitudes toward individual methods. Social media–based information gathering can be used to survey the attitudes of reproductive-age people at a large scale. Social media data could inform healthcare providers’ conversations with patients, be leveraged to improve consumer education, and correct prevalent misconceptions about contraception.