unpywall is a Python client that utilizes the Unpaywall REST API for scholarly analysis with pandas. This package is influenced by roadoi, a R client that interacts with the Unpaywall API.

You can find more about the Unpaywall service here: https://unpaywall.org/.

The documentation about the Unpaywall REST API is located here: https://unpaywall.org/products/api.

Install from pypi using pip:

pip install unpywall

To use the Unpaywall Service, you need to authenticate yourself. For that, unpywall offers multiple ways for authorizing the client. You can either import UnpywallCredentials which generates an environment variable or you can set the environment variable by yourself. Both methods require an email.

from unpywall.utils import UnpywallCredentials

UnpywallCredentials('[email protected]')

Notice that the environment variable for authentication needs to be called UNPAYWALL_EMAIL.

export [email protected]

unpywall uses the data analysis tool pandas for evaluating information from Unpaywall. By default, the Unpaywall API returns a JSON data structure. However, by using get_df, you can transform that into a pandas DataFrame. This also works with multiple DOIs.

from unpywall import Unpywall

Unpywall.get_df(dois=['10.1038/nature12373', '10.1093/nar/gkr1047'])

#   data_standard  ... best_oa_location.version
#0              2  ...         publishedVersion
#1              2  ...         publishedVersion

#[2 rows x 32 columns]

You can track the progress of your API call by setting the parameter progress to True. This is especially useful for estimating the time required.

Unpywall.get_df(dois=['10.1038/nature12373', '10.1093/nar/gkr1047'],
                progress=True)

#|=========================                        | 50%

The method also allows two options for catching errors (raise and ignore)

Unpywall.get_df(dois=['10.1038/nature12373', '10.1093/nar/gkr1047'],
                errors='ignore')

If you are using Unpaywall to obtain full-text copies of papers for literature mining, you may benefit from the following functions:

You can use the download_pdf_handle method to return a PDF handle for the given DOI.

Unpywall.download_pdf_handle(doi='10.1038/nature12373')

#<http.client.HTTPResponse object at 0x7fd08ef677c0>

To return an URL to a PDF for the given DOI, use get_pdf_link.

Unpywall.get_pdf_link(doi='10.1038/nature12373')

#'https://dash.harvard.edu/bitstream/1/12285462/1/Nanometer-Scale%20Thermometry.pdf'

To return an URL to the best available OA copy, regardless of the format, use get_doc_link.

Unpywall.get_doc_link(doi='10.1016/j.envint.2020.105730')

#'https://doi.org/10.1016/j.envint.2020.105730'

To return a list of all URLS to OA copies, use get_all_links.

Unpywall.get_all_links(doi='10.1038/nature12373')

#['https://dash.harvard.edu/bitstream/1/12285462/1/Nanometer-Scale%20Thermometry.pdf']

You can also directly access all data provided by unpaywall in json format using get_json.

Unpywall.get_json(doi='10.1038/nature12373')

#{'best_oa_location': {'endpoint_id': '8c9d8ba370a84253deb', 'evidence': 'oa repository (via OAI-PMH doi match)', 'host_type': ...

unpywall comes with a command-line-interface that can be used to quickly look up a PDF or to download free full-text articles to your device.

Retrieve the URL of a PDF for a given DOI with the following command.

unpywall link 10.1038/nature12373

If you want to view a PDF in your Browser or on your system use view.

unpywall view 10.1038/nature12373 -m browser

Use download if you want to store a PDF on your machine.

unpywall download 10.1038/nature12373 -f article.pdf -p ./documents

You can always use help to open a description for the provided functions.

unpywall -h

Full documentation is available at https://unpywall.readthedocs.io/.

To install unpywall, along with dev tools, run:

pip install -e '.[dev]'