Turn 8,000,000+ US federal contracts, IDVs, grants, and loans into an outbound-ready B2G intelligence database โ with recipient UEI, full headquarters address, congressional district, 3-tier NAICS hierarchy, set-aside program, and FFATA executive compensation.
**Turn US federal procurement data into a ready-to-use B2G lead & market intelligence database.** This **USAspending scraper** extracts federal **contracts, IDVs (Indefinite Delivery Vehicles / IDIQs / GWACs), grants, loans, direct payments, and other financial assistance** from [USAspending.gov](https://www.usaspending.gov) โ complete with the winning company's **Unique Entity ID (`UEI`), parent UEI, full registered address, business categories (Small Business, 8(a), WOSB, HUBZone, SDVOSB), 3-tier NAICS hierarchy, PSC code, set-aside classification, competition metrics, subaward totals, place of performance, awarding & funding agency hierarchy, and disclosed FFATA executive compensation**. No login, no API key, no proxy required. Export to **JSON, CSV, Excel (XLSX), or API**. > ๐ฆ Built for **B2G & GovCon sales teams, GovTech founders, M\&A and private-equity analysts, defense/healthcare/energy strategists, small-business set-aside contractors (8(a) / WOSB / SDVOSB / HUBZone), compliance officers, and investigative researchers** who want structured federal-spending intelligence at a fraction of the cost of GovWin IQ, Bloomberg Government, or Deltek. *** ### โก At a glance | Feature | Details | | :--- | :--- | | ๐๏ธ **All 6 award categories** | `contracts`, `idvs` (IDIQ/GWAC/BPA), `grants`, `loans`, `other_financial_assistance`, and `direct_payments` | | ๐ฏ **Precision B2G targeting** | Filter by Keywords, Awarding Agency (30+ top-tier agencies), Recipient Name, NAICS (built-in + custom), Set-Aside Program (19 codes), HQ State, Performance State, Date Range, and Dollar Amount | | ๐ข **Recipient & UEI enrichment** | `recipient.name`, `recipient.uei`, `parentName`, `parentUei`, `businessCategories[]`, and full HQ address (`addressLine1`, `city`, `stateCode`, `zip5`/`zip4`, `county`, `congressionalCode`) | | ๐ **Executive compensation** | Disclosed FFATA top officers (`executiveCompensation`: `[{ name, amount }]`) with automatic null-slot stripping | | ๐ **Procurement & competition** | `awardAmount`, `totalObligation`, `baseAndAllOptions`, `totalOutlay`, `naics` (2/4/6-digit hierarchy), `psc`, `setAside`, `competition` (`extentCompeted`, `numberOfOffers`, `solicitationProcedures`) | | ๐ **Unlocked 1,000+ pagination** | Paginates `100 awards/page` with parallel detail enrichment (`maxConcurrency` up to `20`) and zero artificial caps when `maxRecords: 1000+` is set | | ๐ **Dual dataset views** | Pre-built **Awards** table view (1 row per award) + **Recipients + executives** unwound view (1 row per disclosed executive) | | ๐พ **Export & CRM sync** | Bulk export JSON, CSV, Excel, or push via Apify API into Salesforce, HubSpot, Clay, Apollo, Smartlead, Make, Zapier, and n8n | *** ### ๐ Why scrape USAspending? The United States federal government is the largest single buyer of goods, services, R\&D, and infrastructure on Earth โ obligating over **$7 trillion annually** across defense, cybersecurity, cloud computing, healthcare, construction, energy, and scientific research. Every contract modification, grant award, and IDV task order is recorded on USAspending.gov. However, the web interface is designed for manual lookup rather than revenue operations or data engineering: 1. **Buried entity & executive fields:** Search result tables omit the recipient's **full street address**, **parent UEI**, **business category flags**, **competition offer counts**, and **FFATA executive compensation** (`officers` names and salaries) โ those require fetching each individual `/awards/{internalId}/` detail endpoint. 2. **Awkward bulk exports:** Official bulk downloads dump massive, multi-gigabyte zipped CSVs with hundreds of raw treasury columns that require hours of ETL cleaning just to answer *"Which 8(a) or SDVOSB cybersecurity contractors in Virginia won DoD contracts over $1M this year?"* This **USAspending scraper** bridges that gap: it queries the official search API with your exact ICP filters, paginates cleanly across hundreds or thousands of awards (`100 per page`), fetches the deep per-award detail record in parallel, and outputs both a clean nested JSON structure and flat CRM-ready columns (`recipientName`, `recipientUei`, `recipientState`, `agencyName`, `naicsCode`, `setAsideDescription`, `performanceState`). **Common reasons teams scrape USAspending:** - ๐ฏ **B2G & GovCon outbound lead generation** โ build targeted lists of federal prime contractors by NAICS, agency, state, and set-aside status, complete with `UEI`, physical HQ address, and named executives (`executiveCompensation`). - ๐ค **Subcontracting & teaming partner discovery** โ small businesses (`8(a)`, `WOSB`, `SDVOSB`, `HUBZone`) identify large primes winning contracts with high `baseAndAllOptions` ceilings and active `subawardCount` activity. - ๐ **Competitor contract-win monitoring** โ schedule weekly runs filtered by `recipientSearchText` (e.g. `"Booz Allen"`, `"Leidos"`, `"Palantir"`) or `naicsCodes` to track competitor obligations and agency expansion. - ๐ผ **M\&A & private-equity deal sourcing** โ screen privately held defense, IT, and engineering contractors by obligated federal revenue (`awardAmount`), contract backlog (`baseAndAllOptions` minus `totalOutlay`), and sole-source moat (`competition.extentCompeted`). - ๐งช **Grant & non-dilutive R\&D intelligence** โ pull NIH, NSF, DOE, DoD SBIR/STTR, and USDA grants (`awardTypeCategories: ["grants"]`) to find funded biotech, climate-tech, and university labs. *** ### โจ What this USAspending scraper does - โ **Covers all 6 federal spending categories** โ `contracts`, `idvs`, `grants`, `loans`, `other_financial_assistance`, and `direct_payments`. - โ **12 combinable ICP filter dimensions** โ filter by `keywords`, `recipientSearchText`, `agencyNames` (30 top-tier federal agencies), `naicsCodes` (31 popular 6-digit codes + `naicsCodesCustom` for any 2/4/6-digit code), `setAsideTypes` (19 small-business set-aside codes), `recipientLocationStates`, `placeOfPerformanceStates`, `timePeriodStart`/`End`, and `minAwardAmount`/`maxAwardAmount`. - โ **Unlocked 1,000+ pagination** โ steps through `spending_by_award` at `100 records/page` (`page=1, 2, 3โฆ`) with zero hard caps when `maxRecords` is set to `1000`, `5000`, or higher. - โ **Deep per-award detail enrichment (`scrapeAwardDetails: true`)** โ fetches `/api/v2/awards/{internalId}/` in parallel to enrich every row with UEI, parent UEI, full address, congressional district, business categories, 3-tier NAICS hierarchy, PSC code, set-aside details, competition metrics, subaward totals, and FFATA executive compensation. - โ **Fast listing-only mode (`scrapeAwardDetails: false`)** โ skips per-award detail requests to stream summary rows 10ร faster for massive industry discovery sweeps. - โ **Clean FFATA executive array** โ strips empty/null officer padding slots so `executiveCompensation` contains only real disclosed executives (`[{ name, amount }]`) or `[]`. - โ **No proxy or API key required** โ runs directly against public federal endpoints with configurable `maxConcurrency` (`1`โ`20`) and `requestDelay`. *** ### ๐ฏ Use cases | Persona | How they use USAspending Federal Awards Scraper | | :--- | :--- | | ๐ **B2G & GovCon Sales Teams** | Prospect companies that just won a $500Kโ$50M federal contract (funded buyers with immediate hiring, software, and compliance budgets). | | ๐ก๏ธ **Small-Business Set-Aside Firms** | Filter by `setAsideTypes: ["8A", "SDVOSBC", "WOSB", "HUBZ"]` and `naicsCodes` to see which agencies and offices are actively awarding set-aside contracts. | | ๐ฆ **Private Equity & M\&A Analysts** | Map market share, contract renewal dates (`periodOfPerformance.currentEndDate`), and disclosed C-suite pay (`executiveCompensation`) across niche NAICS codes. | | ๐งฌ **B2B Biotech & Lab Suppliers** | Scrape HHS / NIH / NSF `grants` to target freshly funded principal investigators and research institutions. | | ๐ฐ **Journalists & Policy Researchers** | Audit federal spending flows by state (`placeOfPerformanceStates`), congressional district (`congressionalCode`), and sole-source competition status. | *** ### ๐ฅ Input Every input field is optional. With default prefill (`awardTypeCategories: ["contracts"]`, `keywords: ["cybersecurity"]`, `maxRecords: 50`), a run completes in **~10โ15 seconds**. Running with an empty `{}` input returns the top 100 federal contracts from the last fiscal year sorted by award amount. | Field | Type | Default | Description | | :--- | :--- | :--- | :--- | | `awardTypeCategories` | array | `["contracts"]` | Categories to extract: `contracts`, `idvs`, `grants`, `loans`, `other_financial_assistance`, `direct_payments`. | | `keywords` | array | `[]` (`prefill: ["cybersecurity"]`) | Free-text keywords matched across award descriptions (e.g. `["cloud", "zero trust", "AI"]`). | | `recipientSearchText` | string | `""` | Contractor or grantee company name substring (e.g. `"Lockheed"`, `"Booz Allen"`, `"Palantir"`). | | `agencyNames` | array | `[]` (All agencies) | Pick one or more of 30 top-tier awarding agencies (`Department of Defense`, `Department of Veterans Affairs`, `NASA`, etc.). | | `naicsCodes` | array | `[]` | Pick from 31 built-in 6-digit NAICS industry codes (`541512` Computer Systems Design, `541330` Engineering, etc.). | | `naicsCodesCustom` | array | `[]` | Any additional 2-, 4-, or 6-digit NAICS codes to combine with `naicsCodes`. | | `setAsideTypes` | array | `[]` | Small-business set-aside codes (`SBA`, `8A`, `8AN`, `WOSB`, `EDWOSB`, `HUBZ`, `SDVOSBC`, `SDVOSBS`, `VSA`, etc.). | | `recipientLocationStates` | array | `[]` | Filter by contractor headquarters US state/territory (`VA`, `MD`, `DC`, `CA`, `TX`, etc.). | | `placeOfPerformanceStates` | array | `[]` | Filter by US state/territory where the work is performed. | | `timePeriodStart` | string | Last fiscal year | Start date (`YYYY-MM-DD`, prefilled `2024-10-01`). | | `timePeriodEnd` | string | Today | End date (`YYYY-MM-DD`). | | `minAwardAmount` | integer | `0` | Minimum award obligation amount in USD (`0` = no floor). | | `maxAwardAmount` | integer | `0` | Maximum award obligation amount in USD (`0` = no ceiling). | | `sortBy` | string | `"Award Amount"` | Sort field: `"Award Amount"`, `"Start Date"`, `"End Date"`, `"Recipient Name"`, or `"Award ID"`. | | `sortOrder` | string | `"desc"` | `"desc"` (largest/newest first) or `"asc"`. | | `scrapeAwardDetails` | boolean | `true` | Fetch the full per-award detail record (UEI, HQ address, NAICS hierarchy, competition, executives). Set `false` for fast summary discovery. | | `maxRecords` | integer | `100` (`prefill: 50`) | Maximum awards to collect. Set `1000`โ`10000+` for deep sweeps, or `0` for unlimited (when filters are active). | | `requestDelay` | integer | `250` | Delay in milliseconds between requests (`250` ms default). | | `maxConcurrency` | integer | `5` | Parallel detail-fetching workers (`1`โ`20`). | | `proxyConfiguration` | object | Off | Optional Apify Proxy configuration. Not needed for standard runs (`useApifyProxy: false`). | #### Example input โ 1,000+ DoD IT & cybersecurity set-aside contracts ```json { "awardTypeCategories": ["contracts"], "agencyNames": ["Department of Defense", "Department of Homeland Security"], "naicsCodes": ["541512", "541511", "541519"], "setAsideTypes": ["8A", "SDVOSBC", "WOSB", "HUBZ"], "recipientLocationStates": ["VA", "MD", "DC"], "timePeriodStart": "2024-10-01", "minAwardAmount": 250000, "sortBy": "Award Amount", "sortOrder": "desc", "scrapeAwardDetails": true, "maxRecords": 1500, "maxConcurrency": 10 } ``` *** ### ๐ค Output Each award is pushed to the dataset with both rich nested objects and flat top-level convenience fields. Below is a real record from a verified run: ```json { "awardId": "HT940216C0001", "internalAwardId": "CONT_AWD_HT940216C0001_9700_-NONE-_-NONE-", "category": "contract", "type": "D", "typeDescription": "DEFINITIVE CONTRACT", "description": "MANAGED CARE SUPPORT SERVICES", "awardAmount": 51269205263.03, "totalObligation": 51269205263.03, "baseAndAllOptions": 56620536577.19, "totalOutlay": 42150112000.0, "dateSigned": "2016-07-29", "periodOfPerformance": { "startDate": "2016-08-01", "currentEndDate": "2025-12-31", "potentialEndDate": "2025-12-31 00:00:00" }, "awardingAgency": { "toptierName": "Department of Defense", "toptierCode": "097", "toptierAbbrev": "DOD", "subtierName": "Defense Health Agency", "subtierCode": "97DH", "officeName": "DEFENSE HEALTH AGENCY" }, "fundingAgency": { "toptierName": "Department of Defense", "subtierName": "Defense Health Agency" }, "recipient": { "name": "HUMANA GOVERNMENT BUSINESS INC", "uei": "ZE6ZM6NKSV43", "parentName": "HUMANA INC", "parentUei": "ZE6ZM6NKSV43", "businessCategories": [ "Category Business", "Corporate Entity Not Tax Exempt", "Not Designated a Small Business", "U.S.-Owned Business" ], "location": { "addressLine1": "500 W MAIN STREET", "addressLine2": null, "city": "LOUISVILLE", "state": "KENTUCKY", "stateCode": "KY", "country": "UNITED STATES", "countryCode": "USA", "zip5": "40202", "zip4": "2946", "county": "JEFFERSON", "congressionalCode": "03" } }, "placeOfPerformance": { "city": "LOUISVILLE", "state": "KENTUCKY", "stateCode": "KY", "country": "UNITED STATES", "countryCode": "USA", "county": "JEFFERSON", "zip5": "40201", "congressionalCode": "03" }, "naics": { "code": "524114", "description": "DIRECT HEALTH AND MEDICAL INSURANCE CARRIERS", "midtierCode": "5241", "midtierDescription": "Insurance Carriers", "toptierCode": "52", "toptierDescription": "Finance and Insurance" }, "psc": { "code": "Q201", "description": "SUPPORT- MEDICAL: HEALTHCARE" }, "setAside": { "code": "NONE", "description": "NO SET ASIDE USED." }, "competition": { "extentCompeted": "A", "extentCompetedDescription": "FULL AND OPEN COMPETITION", "numberOfOffers": 4, "solicitationProcedures": "NP" }, "subawardCount": 12, "subawardTotalAmount": 18450000.0, "executiveCompensation": [ { "name": "KAREN MORAN", "amount": 1409718 }, { "name": "KEVIN REGENHOLD", "amount": 607266 }, { "name": "MELANIE BUEHLER", "amount": 604450 } ], "recipientName": "HUMANA GOVERNMENT BUSINESS INC", "recipientUei": "ZE6ZM6NKSV43", "recipientState": "KY", "agencyName": "Department of Defense", "naicsCode": "524114", "naicsDescription": "DIRECT HEALTH AND MEDICAL INSURANCE CARRIERS", "setAsideDescription": "NO SET ASIDE USED.", "performanceState": "KY", "awardDetailUrl": "https://www.usaspending.gov/award/CONT_AWD_HT940216C0001_9700_-NONE-_-NONE-/", "scrapedAt": "2026-11-10T03:00:00.000Z" } ``` #### ๐ Output field reference ##### ๐๏ธ Award Identity, Financials & Timeline | Field | Type | Coverage | Description | | :--- | :--- | :--- | :--- | | `awardId` | string | ๐ข High (100%) | Official procurement instrument ID (`PIID` for contracts/IDVs, `FAIN`/`URI` for grants/loans). | | `internalAwardId` | string | ๐ข High (100%) | Unique USAspending generated award ID (`generated_unique_award_id`, primary key). | | `category` / `type` / `typeDescription` | string | ๐ข High (100%) | Award category (`contract`, `idv`, `grant`) and subtype (`DEFINITIVE CONTRACT`, `BPA CALL`, `PROJECT GRANT`). | | `description` | string | ๐ข High (~97%) | Scope of work / procurement description. | | `awardAmount` / `totalObligation` | number | ๐ข High (100%) | Total federal dollars obligated on the award. | | `baseAndAllOptions` | number | ๐ข High (~95%) | Total potential contract ceiling including all option years. | | `totalOutlay` | number | ๐ก Conditional (~51%) | Actual cash outlays disbursed to date (populated once treasury outlays post). | | `dateSigned` | string | ๐ข High (100%) | Date the initial award was signed (`YYYY-MM-DD`). | | `periodOfPerformance` | object | ๐ข High (100%) | `{ startDate, currentEndDate, potentialEndDate }`. | | `subawardCount` / `subawardTotalAmount` | number | ๐ข High (~95%) | Total count and dollar volume of reported subcontracts/subgrants. | | `awardDetailUrl` | string | ๐ข High (100%) | Direct URL to the award profile on `usaspending.gov`. | ##### ๐ข Recipient Entity, UEI, HQ Location & Executives | Field | Type | Coverage | Description | | :--- | :--- | :--- | :--- | | `recipient.name` / `recipientName` | string | ๐ข High (100%) | Legal name of the winning contractor or grantee. | | `recipient.uei` / `recipientUei` | string | ๐ข High (~99%) | 12-character SAM.gov Unique Entity Identifier (`UEI`). | | `recipient.parentName` / `parentUei` | string | ๐ข High (~92%) | Ultimate parent company name and parent UEI. | | `recipient.businessCategories` | array of strings | ๐ข High (~99%) | SAM.gov socio-economic and entity classifications (e.g. `Small Business`, `Women Owned Business`, `Veteran Owned Business`). | | `recipient.location` | object | ๐ข High (~99%) | Full HQ address: `addressLine1`, `addressLine2`, `city`, `state`, `stateCode`, `zip5`, `zip4`, `county`, `countryCode`, `congressionalCode`. | | `executiveCompensation` | array of objects | ๐ก Conditional (~35%) | Array of `{ name, amount }` for top 5 officers when the contractor meets FFATA disclosure thresholds (`[]` otherwise). | ##### ๐ฏ Agency, NAICS, PSC, Set-Aside & Competition | Field | Type | Coverage | Description | | :--- | :--- | :--- | :--- | | `awardingAgency` | object | ๐ข High (100%) | `{ toptierName, toptierCode, toptierAbbrev, subtierName, subtierCode, officeName }`. | | `fundingAgency` | object | ๐ข High (~98%) | `{ toptierName, subtierName }` providing the treasury funds. | | `placeOfPerformance` | object | ๐ข High (~99%) | `{ city, state, stateCode, county, zip5, countryCode, congressionalCode }` where work is executed. | | `naics` | object | ๐ข High (~98%) | 3-tier NAICS hierarchy: `code`/`description` (6-digit), `midtierCode`/`midtierDescription` (4-digit), `toptierCode`/`toptierDescription` (2-digit). | | `psc` | object | ๐ข High (~98%) | `{ code, description }` Federal Product or Service Code. | | `setAside` / `setAsideDescription` | object / string | ๐ข High (~86%) | Small-business set-aside code and description (for contracts). | | `competition` | object | ๐ข High (~95%) | `{ extentCompeted, extentCompetedDescription, numberOfOffers, solicitationProcedures }`. | *** ### ๐ Pagination & 1,000+ scale - **Unlocked 1,000+ discovery pagination:** The scraper POSTs to `/api/v2/search/spending_by_award/` at `100 records per page` (`page=1, 2, 3โฆ`), following `page_metadata.hasNext` until `discovered.length >= maxRecords` or all matching awards are retrieved. - **Honors explicit `maxRecords` on unfiltered runs:** Even if you leave all narrowing filters blank and set `maxRecords: 1000`, the scraper honors your explicit `1000` limit rather than clamping down to `100` (the 100-award sample cap only applies when `maxRecords === 0` on a completely unfiltered run). - **Parallel detail worker pool:** Discovered awards are enriched concurrently (`maxConcurrency` default `5`, adjustable up to `20`), completing `50` detailed awards in **~12 seconds** and `500` detailed awards in **~35 seconds**. *** ### โ๏ธ How to use 1. Open **USAspending Federal Awards Scraper** in the **Apify Console**. 2. Choose your **Award types** (`contracts`, `idvs`, `grants`, etc.) and set any targeting filters (**Awarding agencies**, **Industry NAICS**, **Small-business set-asides**, **Contractor location states**, **Keywords**, or **Minimum award amount**). 3. Keep **Include full award details** (`scrapeAwardDetails: true`) enabled for UEI, address, and executive compensation, or disable it for ultra-fast summary sweeps. 4. Set **Max results** (`maxRecords`: `50` prefilled for a fast `<15s` test run, or `1000+` for full pipeline extraction). 5. Click **Start** and export from the **Awards** or **Recipients + executives** dataset views to **CSV, Excel, or JSON**. #### Run via Apify API (cURL) ```bash curl -X POST "https://api.apify.com/v2/acts/haketa~usaspending-scraper/runs?token=YOUR_APIFY_TOKEN" \ -H "Content-Type: application/json" \ -d '{ "awardTypeCategories": ["contracts"], "agencyNames": ["Department of Defense"], "naicsCodes": ["541512"], "minAwardAmount": 500000, "scrapeAwardDetails": true, "maxRecords": 1000, "maxConcurrency": 10 }' ``` *** ### ๐ Programmatic usage (Python & Node.js) #### Python (`apify-client`) ```python from apify_client import ApifyClient client = ApifyClient("YOUR_APIFY_TOKEN") run_input = { "awardTypeCategories": ["contracts"], "agencyNames": ["Department of Defense"], "naicsCodes": ["541512", "541330"], "setAsideTypes": ["8A", "SDVOSBC", "WOSB"], "minAwardAmount": 250000, "scrapeAwardDetails": True, "maxRecords": 1000, } run = client.actor("haketa/usaspending-scraper").call(run_input=run_input) for award in client.dataset(run["defaultDatasetId"]).iterate_items(): print(f"{award.get('awardId')} | {award.get('recipientName')} (UEI: {award.get('recipientUei')}) | ${award.get('awardAmount')}") ``` #### Node.js (`apify-client`) ```javascript import { ApifyClient } from 'apify-client'; const client = new ApifyClient({ token: 'YOUR_APIFY_TOKEN' }); const run = await client.actor('haketa/usaspending-scraper').call({ awardTypeCategories: ['contracts', 'grants'], keywords: ['cybersecurity', 'artificial intelligence'], scrapeAwardDetails: true, maxRecords: 500, }); const { items } = await client.dataset(run.defaultDatasetId).listItems(); console.log(`Extracted ${items.length} federal awards:`, items[0]); ``` *** ### ๐ก Tips for best results - ๐ **Use `recipientUei` as your master B2G join key:** Every federal contractor has a unique 12-character SAM.gov `UEI`. Join `recipientUei` with SAM.gov entity data, SBA Dynamic Small Business Search, or Clay/Apollo enrichment. - ๐ **Switch to the "Recipients + executives" dataset tab:** The built-in `recipients` dataset view automatically unwinds `executiveCompensation` so each row is a named C-suite officer with their declared compensation and company UEI. - โก **Tune `maxConcurrency: 10` and `requestDelay: 100` for 1,000+ detail runs:** The USAspending API handles 10 concurrent detail fetches smoothly, cutting 1,000-record run times to under 60 seconds. *** ### โ ๏ธ Honest limitations - **FFATA executive compensation threshold:** Under the Federal Funding Accountability and Transparency Act (FFATA), contractors are only required to disclose top-5 executive pay if โฅ80% and โฅ$25M of their revenue comes from federal awards and compensation isn't already public via SEC filings. Consequently, `executiveCompensation` is populated on ~25โ60% of awards and `[]` on the rest. - **Assistance vs. procurement fields:** Grants and loans (`grants`, `loans`) use `FAIN`/`URI` instead of `PIID` and do not carry contract-specific `setAside`, `psc`, or `competition.numberOfOffers` fields. *** ### โ FAQ **How many federal awards can I scrape in a single run?** Tens of thousands. Set `maxRecords: 1000`, `5000`, or `10000+` โ the scraper paginates 100 awards per page until your target is reached. **Do I need an API key, SAM.gov account, or proxy?** No. USAspending.gov endpoints are public-domain US government infrastructure and run cleanly without API keys or proxies. **Why should I keep `scrapeAwardDetails: true` enabled?** The summary search endpoint only returns 14 basic columns (no UEI, no street address, no business categories, no competition offer counts, and no executive compensation). Enabling `scrapeAwardDetails: true` fetches the full `/api/v2/awards/{id}/` record for every award. **Are there duplicate awards in the output?** No. Every award is keyed by its unique `generated_internal_id` (`internalAwardId`), ensuring 0% duplicates across pages. *** ### ๐ Related scrapers - [EPA ECHO Facility Compliance Scraper](https://apify.com/haketa/epa-echo-scraper) โ US facility inspections, violations, and enforcement history. - [NPI Registry Scraper (7M+ US Healthcare Providers)](https://apify.com/haketa/nppes-npi-registry-scraper) โ US healthcare providers and organizations by NPI, taxonomy, and state. - [H1B Visa Database Scraper](https://apify.com/haketa/h1b-visa-database-scraper) โ DOL H1B visa sponsor employers and salary disclosures. *** ### ๐ Changelog #### v1.2 (2026-11) โ Unlocked 1,000+ Unfiltered Pagination & Fast Prefill - **Unlocked 1,000+ unfiltered runs:** Fixed `NO_FILTER_RUN` logic in `src/main.js` so explicit user `maxRecords > 100` (e.g. `maxRecords: 1000`) is always honored even when no narrowing filter is set; the 100-award sample guard now triggers only when `maxRecords === 0` on an unfiltered run. - **Fast prefill execution:** Tuned `maxRecords` prefill to `50` so full-detail Console test runs finish in **~12 seconds**. #### v1.1 (2026-10) โ Executive Compensation Unwind View & NAICS Hierarchy - Added the `recipients` dataset view unwinding `executiveCompensation` into one row per disclosed officer. - Added 3-tier `naics` hierarchy (`code`, `midtierCode`, `toptierCode`) and stripped null officer slots. #### v1.0 (2026-09) โ Initial Release - Initial release supporting all 6 federal award categories, 12 filter dimensions, and parallel `/api/v2/awards/` detail enrichment. # Changelog This Actor's version history is a separate document: https://apify.com/haketa/usaspending-scraper/changelog.md # Actor input Schema ## `awardTypeCategories` (type: `array`): Which categories of federal spending to pull. Pick one or more. Leave as-is to focus on contracts. ## `keywords` (type: `array`): Optional free-text search across award descriptions. Great for finding specific topics such as cybersecurity, cloud migration, AI, or logistics. Leave empty to browse everything. ## `recipientSearchText` (type: `string`): Optional. Type part of a contractor / company name to find every award they won (for example Lockheed, Booz Allen, or Microsoft). ## `agencyNames` (type: `array`): Pick one or more federal agencies to focus on. Leave empty to include every agency. ## `naicsCodes` (type: `array`): Pick one or more industries to focus on. Leave empty to include every industry. ## `naicsCodesCustom` (type: `array`): Advanced: add any other NAICS industry codes not in the list above (2 / 4 / 6 digit). Combined with the picks above. ## `setAsideTypes` (type: `array`): Focus on awards reserved for specific small-business programs. Leave empty to include all awards. ## `recipientLocationStates` (type: `array`): Only include contractors headquartered in these states. Leave empty for nationwide. ## `placeOfPerformanceStates` (type: `array`): Only include awards performed in these states. Leave empty for nationwide. ## `timePeriodStart` (type: `string`): Only include awards active on/after this date. Leave empty to use the last full fiscal year. ## `timePeriodEnd` (type: `string`): Only include awards active on/before this date. Leave empty for today. ## `minAwardAmount` (type: `integer`): Only include awards worth at least this much. 0 = no minimum. ## `maxAwardAmount` (type: `integer`): Only include awards worth at most this much. 0 = no maximum. ## `sortBy` (type: `string`): How to order the results. ## `sortOrder` (type: `string`): Show largest/latest first, or smallest/oldest first. ## `scrapeAwardDetails` (type: `boolean`): Pull the full record for each award โ recipient UEI, full address, NAICS hierarchy, set-aside type, executive compensation, place of performance and more. Turn OFF for ultra-fast bulk discovery. ## `maxRecords` (type: `integer`): Maximum number of awards to return. 0 = unlimited (millions of awards exist, so a limit is recommended). ## `requestDelay` (type: `integer`): Politeness delay between requests. ## `maxConcurrency` (type: `integer`): How many award records to fetch in parallel. ## `proxyConfiguration` (type: `object`): Optional. Runs fine without a proxy โ only enable one if you are running very large volumes. ## Actor input object example ```json { "awardTypeCategories": [ "contracts" ], "keywords": [ "cybersecurity" ], "recipientSearchText": "", "agencyNames": [], "naicsCodes": [], "naicsCodesCustom": [], "setAsideTypes": [], "recipientLocationStates": [], "placeOfPerformanceStates": [], "timePeriodStart": "2024-10-01", "timePeriodEnd": "", "minAwardAmount": 0, "maxAwardAmount": 0, "sortBy": "Award Amount", "sortOrder": "desc", "scrapeAwardDetails": true, "maxRecords": 50, "requestDelay": 250, "maxConcurrency": 5, "proxyConfiguration": { "useApifyProxy": false } } ``` # Actor output Schema ## `dataset` (type: `string`): Open to view, filter and export all scraped results from this run (JSON, CSV, Excel). # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '