secfile: SEC EDGAR Filings in R and Python

Open Source Quantitative Finance

Overview

secfile is a package that provides simple and efficient access to the SEC’s EDGAR APIs (https://www.sec.gov/search-filings) for querying and retrieving filings.

The core functionality of the secfile package abstracts the complexities of interacting with SEC EDGAR APIs, such as session management, user agent declaration, rate limiting, index parsing, pagination of filing metadata, URL construction, document caching, and inline XBRL parsing. This abstraction allows users to focus on retrieving data rather than managing API details. Use cases include retrieving data across a range of workflows:

• Indexes: master index of all filings by form type and date for universe construction
• Tenures: status windows built by pairing entry and exit form filings
• Submissions: filing metadata for any filer with form type and date range filters
• Facts: investment-level or company-level facts extracted from inline XBRL filings

The package supports flexible query capabilities, including customizable form types, date ranges, and dimensions, and validates the retrieved data automatically. It handles the SEC’s fair access requirements, such as user agent declaration and rate limiting between requests, and caches downloaded documents for efficient retrieval of large datasets.

The implementation uses standard HTTP libraries to handle API interactions efficiently and supports both R and Python to make it accessible for a broad audience.

Installation in R

• Install the released version from CRAN:

install.packages("secfile")

• Or the development version from GitHub:

# install.packages("pak")
pak::pak("jasonjfoster/file/r")

• Then load the package:

library(secfile)

Installation in Python

• Install the released version from PyPI:

pip install secfile

• Or the development version from GitHub:

pip install \
  git+https://github.com/jasonjfoster/file.git@main#subdirectory=python

• Then import the package:

import secfile as sec

Available forms

Load the package and explore the available form types using secfile::data_forms in R or sec.data_forms in Python, and the available current report (“8-K”) item numbers using secfile::data_items in R or sec.data_items in Python:

The SEC requires a user agent that declares contact information (e.g., “username@domain.com”) for fair access, so pass the user_agent argument to identify the user. Alternatively, pass session = get_session(user_agent) to reuse a connection across function calls.

Usage

Get CIKs

The secfile::get_ciks function in R and sec.get_ciks function in Python get the Central Index Key (“CIK”) for one or more tickers from the SEC EDGAR APIs.

Parameters

• tickers: string. Ticker or vector of tickers to filter, or NULL (None in Python) for all tickers.
• user_agent: string. User agent with contact information.
• session: list. Session created using the get_session() function. When a session is provided, the user_agent argument is ignored.

Value

A data frame that contains the company, CIK, and ticker for each filer.

Examples

• Company fundamentals: retrieve annual report filings for one or more known filers, then extract company-level facts from the inline XBRL filings.

R

user_agent <- "username@domain.com"
ciks <- secfile::get_ciks(c("AAPL", "MSFT"), user_agent = user_agent)

Python

user_agent = "username@domain.com"
ciks = sec.get_ciks(["AAPL", "MSFT"], user_agent = user_agent)

Get submissions

The secfile::get_submissions function in R and sec.get_submissions function in Python get the filing metadata (“submissions”) from the SEC EDGAR APIs for one or more filers with optional form type and date range filters.

Parameters

• ciks: string, numeric, or data frame. CIK or vector of CIKs, or a data frame created using get_ciks() or create_tenures() that contains a cik column with optional start_date and end_date columns.
• forms: string. Form type or vector of form types to filter (see data_forms), or NULL (None in Python) for all form types.
• from_date: string. Start date in “YYYY-MM-DD” format.
• to_date: string. End date in “YYYY-MM-DD” format.
• user_agent: string. User agent with contact information.
• session: list. Session created using the get_session() function. When a session is provided, the user_agent argument is ignored.

Value

A data frame that contains the filing metadata for the specified filer(s) with the archives URL for each filing.

Examples

R

submissions <- secfile::get_submissions(ciks, forms = "10-K",
                                        from_date = "2023-01-01",
                                        user_agent = user_agent)

Python

submissions = sec.get_submissions(ciks, forms = "10-K",
                                  from_date = "2023-01-01",
                                  user_agent = user_agent)

Get data

The secfile::get_data function in R and sec.get_data function in Python get facts from inline XBRL filings from the SEC EDGAR APIs using the specified filing metadata. By default, the result contains all contexts that match the report date of each filing and includes the period type, start and end dates, and dimension axes and members for each context.

Parameters

• data: data frame. Filing metadata that contains the CIK, accession number, primary document, and report date for each filing created using the get_submissions() function.
• dimension: string. Dimension of contexts to match (i.e., “typed”, “explicit”, or an axis name such as “InvestmentIdentifierAxis”), or NULL (None in Python) for all contexts.
• date: string. Date in “YYYY-MM-DD” format to match context periods, or NULL (None in Python) for the report date of each filing. Instant contexts match when the instant equals the date and duration contexts match when the end date equals the date.
• cache_dir: string. Directory to cache downloaded XBRL instance documents, or NULL (None in Python) to disable caching.
• user_agent: string. User agent with contact information.
• session: list. Session created using the get_session() function. When a session is provided, the user_agent argument is ignored.

Value

A data frame that contains facts from the SEC EDGAR APIs for the specified filing metadata with the period type, start and end dates, and dimension axes and members for each context.

Examples

R

data <- secfile::get_data(submissions, cache_dir = "cache",
                          user_agent = user_agent)

Python

data = sec.get_data(submissions, cache_dir = "cache",
                    user_agent = user_agent)

View data

Company-level totals are the contexts without dimensions (i.e., the rows where axis is empty). Balance-sheet facts are reported for a point in time, so filter the period_type column to “instant” contexts (duration contexts carry income-statement facts instead):

R

totals <- data[which((data[["axis"]] == "") &
                     (data[["period_type"]] == "instant")), ]
head(totals[ , c("cik", "report_date", "Assets",
                 "Liabilities", "StockholdersEquity", ...)])

Python

totals = data[(data["axis"] == "") & (data["period_type"] == "instant")]
totals[["cik", "report_date", "Assets",
        "Liabilities", "StockholdersEquity", ...]].head()

More workflows in R

Each workflow defines the form types and date range, then follows the same steps to get the session and get the index, or to get the CIKs and get the submissions:

• Filing index: build a universe of filings from the master index by form type and date for bulk download or historical research.

session <- secfile::get_session(user_agent)
index <- secfile::get_index(from_year = 2024, to_year = 2024,
                            forms = "10-K", session = session)

• Tenures: pair exchange listing registration (“8-A12B”) and removal filings to determine listing status windows for each filer and construct survivorship-bias-free universes at any date. Removals are filed by the issuer (“25”) or the exchange (“25-NSE”) and occur at any later date, so omit to_year to run the index through the present.

index <- secfile::get_index(from_year = 2024,
                            forms = c("8-A12B", "25", "25-NSE"),
                            session = session)
tenures <- secfile::create_tenures(index, "8-A12B", c("25", "25-NSE"))

More workflows in Python

Each workflow defines the form types and date range, then follows the same steps to get the session and get the index, or to get the CIKs and get the submissions:

• Tenures: pair exchange listing registration (“8-A12B”) and removal (“25” and “25-NSE”) filings to determine listing status windows for each filer.

session = sec.get_session(user_agent)
index = sec.get_index(from_year = 2024,
                      forms = ["8-A12B", "25", "25-NSE"],
                      session = session)
tenures = sec.create_tenures(index, "8-A12B", ["25", "25-NSE"])

• Event monitoring: track material events by retrieving current report (“8-K”) filing metadata for a watchlist of filers over a recent date range.

ciks = sec.get_ciks(["AAPL", "MSFT", "TSLA"], user_agent = user_agent)
submissions = sec.get_submissions(ciks, forms = "8-K",
                                  from_date = "2026-01-01",
                                  user_agent = user_agent)

• Items: filter the item numbers in the items column (see sec.data_items) for specific events, such as results of operations (“2.02”) or officer departures and appointments (“5.02”).

results = submissions[submissions["items"].str.contains("2.02", regex = False)]

Application

Next we use the secfile package to analyze how survivorship bias influences fundamental analysis of U.S. filers. A universe built from filers listed today ignores filers that were delisted along the way, so backtests inherit a selection bias toward survivors. To construct a point-in-time universe, we pair exchange listing registration (“8-A12B”) and removal (“25” and “25-NSE”) filings with the create_tenures function to determine listing status windows for each filer.

We focus on the cohort of filers that listed during 2020–2021, a period with a large share of initial public offerings and special-purpose acquisition companies, for three reasons. First, 2021 is the largest listing year in the sample, with roughly three times the listings of each of the 2016–2019 cohorts. Second, the 2020 and 2021 cohorts show the highest attrition within three years of listing (with 2022 close behind). Third, the cohort is the earliest observable over its entire life with inline XBRL, which phased in between 2019 and 2021. The table below reports listings and attrition by entry year. The delisted (%) column is cumulative as of June 30, 2026, and attrition horizons a cohort has not yet reached are marked “NA” (e.g., the three-year horizon for the 2023 cohort is incomplete until the end of 2026).

For comparison, a tally by Jay Ritter at the University of Florida also peaks in 2021 (https://site.warrington.ufl.edu/ritter/ipo-data/). It reports 413 initial public offerings in 2020 and a record 924 in 2021, with special-purpose acquisition companies accounting for more than half of each year’s total. The tenure counts are higher because exchange listing registration also captures listings outside this tally, such as uplistings from over-the-counter markets, transfers between exchanges, spin-offs, and smaller offerings the tally excludes.

We then retrieve annual report (“10-K”) filings and extract company-level facts from the inline XBRL filings to compare a fundamental metric across two universes at each quarter-end. The metric is leverage, defined as total liabilities divided by total assets from the balance-sheet facts, and the universes are cohort filers listed at that time (“point-in-time”) and cohort filers still listed as of June 30, 2026 (“survivors”). Each universe uses each filer’s most recent annual report within the trailing year, so membership requires both an active tenure and a report in that window. Note that the analysis uses report dates for simplicity. This assumes the facts are available at the report date rather than at the later filing date, and the assumption applies to both universes. The objective is to measure the survivorship bias in the cross-sectional estimate:

where  is the cross-sectional median of the fundamental metric at time . The point-in-time universe includes every cohort filer whose tenure contains time , regardless of subsequent delisting, while the survivor universe conditions on tenures that remain active as of June 30, 2026.

At each date, the bias is the difference in medians, where leverage holds the cross-section of the metric, and survivors and point_in_time index the two universes at that date:

R

bias <- median(leverage[survivors]) - median(leverage[point_in_time])

Python

bias = leverage[survivors].median() - leverage[point_in_time].median()

The two universes draw on the same facts and differ only in membership, so the bias isolates the effect of conditioning on survival rather than differences in measurement. Note that the survivor universe is observable only in hindsight because membership conditions on tenures active as of June 30, 2026, while the point-in-time universe is available at each date.

Results

After the tenures and facts are combined, we compare the cross-sectional estimates over time. The chart below shows the median leverage for the point-in-time and survivor universes. The analysis provides insight into how universe construction, specifically the inclusion or exclusion of delisted filers, affects conclusions drawn from fundamental data.

Data source: SEC EDGAR APIs: https://www.sec.gov/search-filings

The survivor universe shows a higher median than the point-in-time universe from December 2021 through December 2023: the difference opens at the 2021 cohort’s first annual reports and peaks at roughly 22 percentage points on September 30, 2022. The direction is the opposite of the classical expectation that distressed, highly levered filers delist: among the filers that exited, 64% had leverage below 10% at the peak, and median leverage was roughly 8% for exits versus 33% for survivors. The exits were therefore predominantly shells rather than distressed operators. In the tally above, a large share were special-purpose acquisition companies, which hold trust assets with minimal liabilities. These companies typically have about two years, plus extensions, to complete an acquisition before liquidating, a deadline that concentrated the 2021 cohort’s attrition in the second and third years after listing. The two universes converge by construction once delisted filers no longer report within the trailing year. In this cohort, conditioning on surviving filers overstates the cross-sectional median because the filers that later exited had lower leverage than the survivors. The concentration of the 2021 cohort’s exits in the second and third years after listing determines how large the bias becomes and how long it persists. More generally, the sign of the bias depends on where the filers that later exit sit in the leverage distribution relative to survivors.

Conclusion

The secfile package provides simple and efficient access to the SEC’s EDGAR APIs for querying and retrieving filings. It abstracts the complexities of session management, user agent declaration, rate limiting, index parsing, pagination of filing metadata, URL construction, document caching, and inline XBRL parsing. This allows users to focus on retrieving data across a range of workflows, including indexes, tenures, submissions, and facts. The package supports flexible query capabilities, such as customizable form types, date ranges, and dimensions, and handles the SEC’s fair access requirements automatically while caching downloaded documents to retrieve large datasets efficiently. It is available for both R and Python to make it accessible for a broad audience. The analysis of survivorship bias illustrates how the package can be used to construct point-in-time universes and interpret differences in fundamental estimates across universes. The workflow is available in both languages: for more examples, go to https://github.com/jasonjfoster/file/tree/main/r/examples for R code and https://github.com/jasonjfoster/file/tree/main/python/examples for Python code.