Bulk property datasets vs the API: When to use each

Alex Wilkinson

CEO of Houski

2023-06-22

A question we get from data engineering teams almost every week: "Should we buy your bulk property dataset, or just hit the API?"

The honest answer is "it depends on your workload", and this post breaks down the decision. Both products exist for a reason. If you pick wrong, you either pay too much or grind your pipeline to a halt.

The short version

• Use the API if your queries are point lookups, small batches, or interactive. Anything where freshness matters more than throughput.
• Use the bulk dataset if you need to scan millions of properties, train models, run analytics, or join Houski data against your internal data warehouse.
• Use both if you are running a production system that does both batch analytics and live lookups. This is the common pattern.

What the API gives you

The API is built for low-latency, per-request access to property data. You ask for a property by address, a small geographic radius, or a filtered query, and you get JSON back in milliseconds. The block below is a live call against the real API, the request code and the JSON response are generated every time this page is rendered:

const houski_data = async (): Promise<PropertiesResponse> => {

    // You must copy the PropertiesResponse type declarations from the 
    // Houski API documentation to strongly type the response

    const url = new URL('https://api.houski.ca/properties');
    url.searchParams.set('api_key', 'YOUR_API_KEY');
    url.searchParams.set('city', 'calgary');
    url.searchParams.set('country_abbreviation', 'ca');
    url.searchParams.set('province_abbreviation', 'ab');
    url.searchParams.set('select', 'address,property_type,interior_sq_m,bedroom,bathroom_full,assessment_value,estimate_sale_price');

    const response = await fetch(url);
    const data = await response.json();

    return data;
}

(async () => {
let data: PropertiesResponse = await houski_data();

// Log the response
console.log(data);
})();

{
  "cache_hit": false,
  "cost_cents": 3.179999828338623,
  "data": [
    {
      "address": "31 Hawkside Park NW",
      "assessment_value": 648500,
      "bathroom_full": 3,
      "bedroom": 3,
      "estimate_sale_price": 652319,
      "interior_sq_m": 126.34708404541016,
      "property_id": "10000f97f5cb7b9f",
      "property_type": "Duplex"
    },
    {
      "address": "6 1744 7 Street SW",
      "bathroom_full": 2,
      "bedroom": 3,
      "estimate_sale_price": 826706,
      "interior_sq_m": 125.882568359375,
      "property_id": "10004f7afe0c1946",
      "property_type": "House"
    },
    {
      "address": "384 Copperpond Landng SE",
      "bathroom_full": 2,
      "bedroom": 3,
      "estimate_sale_price": 441172,
      "interior_sq_m": 125.882568359375,
      "property_id": "10007f9761f49940",
      "property_type": "House"
    },
    {
      "address": "239 Dalhurst Way NW",
      "assessment_value": 1110000,
      "bathroom_full": 2,
      "bedroom": 3,
      "estimate_sale_price": 1012722,
      "interior_sq_m": 108.46339416503906,
      "property_id": "100086f6bc064d3f",
      "property_type": "House"
    },
    {
      "address": "52 Cedargrove Way SW",
      "assessment_value": 662000,
      "bathroom_full": 1,
      "bedroom": 3,
      "estimate_sale_price": 614848,
      "interior_sq_m": 136.4734344482422,
      "property_id": "1000c277cd905d3b",
      "property_type": "House"
    },
    {
      "address": "28 Sundown Gr SE",
      "assessment_value": 692500,
      "bathroom_full": 3,
      "bedroom": 3,
      "estimate_sale_price": 781636,
      "interior_sq_m": 172.61241149902344,
      "property_id": "1001109ab2aebbc0",
      "property_type": "House"
    }
  ],
  "error": "",
  "pagination": {
    "current_page": 1,
    "has_next_page": true,
    "has_previous_page": false,
    "page_total": 113490
  },
  "price_quote": false,
  "result_total": 680936,
  "time_ms": 76,
  "ui_info": {
    "city": "Calgary",
    "city_id": "6ec95b53075d062c",
    "city_link": "ca/ab/calgary",
    "city_slug": "calgary",
    "country": "Canada",
    "country_abbreviation": "CA",
    "country_abbreviation_id": "9ace2b6431b7f1be",
    "country_abbreviation_link": "ca",
    "country_slug": "canada",
    "province": "Alberta",
    "province_abbreviation": "AB",
    "province_abbreviation_id": "aae1f05a0f89d2c7",
    "province_abbreviation_link": "ca/ab",
    "province_slug": "alberta"
  }
}

Strengths:

• Always fresh. When a municipality publishes new assessment values or a new permit hits a city open data portal, the API reflects it within our refresh cadence (usually hours to days).
• No storage cost on your side. You query when you need data. You do not warehouse 19 million records.
• Simple integration. Any language that can make HTTP calls works.
• Good for sparse workloads. If you only need 1,000 lookups a day, the API is far cheaper than buying a full dataset.

Limits:

• Rate limits and per-call costs add up at scale. Pulling 5 million records via individual API calls is technically possible but slow and expensive.
• Network latency. Each call takes tens to hundreds of milliseconds. For analytic workloads where you want to process 100 million rows, that is the wrong shape.
• No raw bulk export from the API. The API is designed for queries, not dumps.

What the bulk dataset gives you

The bulk dataset is a snapshot of Houski's full property database, delivered as comma-separated values (CSV) or JSON files. You download once, load into your warehouse, and query at the speed of your local compute.

Typical bulk delivery includes:

• A file per province (or a single national file, your choice)
• One row per property, with 200+ standardized columns
• A schema doc explaining every field
• Optional companion files: listings history, building permits, assessment history

Strengths:

• Throughput. Scanning 19 million Canadian properties in DuckDB or BigQuery takes seconds, not weeks of API calls.
• Cheap per-record. When amortized across a real workload, bulk pricing is far lower per property than API pricing.
• Joinable. You can join Houski's data against your internal customer table, loan book, claims history, or whatever else, using SQL in your own warehouse.
• Reproducible. You hold a fixed snapshot. Models trained on it are exactly reproducible months later.

Limits:

• Staleness. A snapshot is frozen on the day you bought it. If a property reroofs next week, your file does not know.
• Storage and compute on your side. You need a warehouse, a Spark cluster, DuckDB, or something equivalent.
• Upfront cost. Bulk pricing is a single larger number, not pay-as-you-go.

The hybrid pattern (what most production teams do)

Most teams we work with end up using both. Here is the pattern:

1. Buy the bulk dataset for cold storage and model training. Load it into your warehouse. Use it for analytics, batch ML training, and historical reporting.
2. Refresh the dataset quarterly or as needed. Re-download the snapshot when you need fresher data for analytics.
3. Use the API for live lookups. When a user enters an address in your app, when a claim comes in, when a loan is funded, hit the API for the up-to-date record.
4. Use the API for new properties. If your dataset is from January and you need a property that was added in March, the API has it.

This pattern gives you the best of both: cheap analytics at scale, fresh data for live decisions.

Which one fits your workload?

Decision matrix.

A concrete dollar example

Say you are building a flood risk model for a Canadian insurer. You want to train on every single-family home in British Columbia and Alberta with the score_flood, construction_year, roof_material, and a few other features. That is roughly 3 million properties.

• Doing it via API: 3 million calls. At even very generous rate limits, this takes days and costs significantly more than the bulk option. You would be making API calls just to feed a training job.
• Doing it via bulk: One export. Load it into DuckDB. Run your training job locally. Total cost is the one-time dataset fee, and the training job runs in minutes.

Then, once your model is in production scoring new policies one address at a time, the API is the right tool. The two products complement each other.

If you want to talk about which fits your shape of workload, our datasets page lists the available bulk datasets, and the quick start guide covers the API. Sign up, poke around, and reach out if you need a custom slice.

Field selection still matters

One thing worth flagging for API users: even when you are doing live lookups, always use ?select= to ask only for the fields you need. Returning the full 200+ column property record on every call wastes bandwidth and slows your app. The block below shows a properly trimmed live call:

const houski_data = async (): Promise<PropertiesResponse> => {

    // You must copy the PropertiesResponse type declarations from the 
    // Houski API documentation to strongly type the response

    const url = new URL('https://api.houski.ca/properties');
    url.searchParams.set('api_key', 'YOUR_API_KEY');
    url.searchParams.set('city', 'calgary');
    url.searchParams.set('country_abbreviation', 'ca');
    url.searchParams.set('province_abbreviation', 'ab');
    url.searchParams.set('select', 'address,bedroom,bathroom_full,interior_sq_m,estimate_sale_price');

    const response = await fetch(url);
    const data = await response.json();

    return data;
}

(async () => {
let data: PropertiesResponse = await houski_data();

// Log the response
console.log(data);
})();

{
  "cache_hit": false,
  "cost_cents": 2.5199999809265137,
  "data": [
    {
      "address": "31 Hawkside Park NW",
      "bathroom_full": 3,
      "bedroom": 3,
      "estimate_sale_price": 652319,
      "interior_sq_m": 126.34708404541016,
      "property_id": "10000f97f5cb7b9f"
    },
    {
      "address": "6 1744 7 Street SW",
      "bathroom_full": 2,
      "bedroom": 3,
      "estimate_sale_price": 826706,
      "interior_sq_m": 125.882568359375,
      "property_id": "10004f7afe0c1946"
    },
    {
      "address": "384 Copperpond Landng SE",
      "bathroom_full": 2,
      "bedroom": 3,
      "estimate_sale_price": 441172,
      "interior_sq_m": 125.882568359375,
      "property_id": "10007f9761f49940"
    },
    {
      "address": "239 Dalhurst Way NW",
      "bathroom_full": 2,
      "bedroom": 3,
      "estimate_sale_price": 1012722,
      "interior_sq_m": 108.46339416503906,
      "property_id": "100086f6bc064d3f"
    },
    {
      "address": "52 Cedargrove Way SW",
      "bathroom_full": 1,
      "bedroom": 3,
      "estimate_sale_price": 614848,
      "interior_sq_m": 136.4734344482422,
      "property_id": "1000c277cd905d3b"
    },
    {
      "address": "28 Sundown Gr SE",
      "bathroom_full": 3,
      "bedroom": 3,
      "estimate_sale_price": 781636,
      "interior_sq_m": 172.61241149902344,
      "property_id": "1001109ab2aebbc0"
    }
  ],
  "error": "",
  "pagination": {
    "current_page": 1,
    "has_next_page": true,
    "has_previous_page": false,
    "page_total": 113490
  },
  "price_quote": false,
  "result_total": 680936,
  "time_ms": 71,
  "ui_info": {
    "city": "Calgary",
    "city_id": "6ec95b53075d062c",
    "city_link": "ca/ab/calgary",
    "city_slug": "calgary",
    "country": "Canada",
    "country_abbreviation": "CA",
    "country_abbreviation_id": "9ace2b6431b7f1be",
    "country_abbreviation_link": "ca",
    "country_slug": "canada",
    "province": "Alberta",
    "province_abbreviation": "AB",
    "province_abbreviation_id": "aae1f05a0f89d2c7",
    "province_abbreviation_link": "ca/ab",
    "province_slug": "alberta"
  }
}

For bulk users, the equivalent is column pruning at warehouse load time. Once the data is loaded, reading only the columns you need is cheap in DuckDB, BigQuery, Snowflake, or any modern engine.

What is in the bulk dataset

For people who have not seen one before, the schema mirrors the API. Same field names, same value standardization, same property IDs. If you build something on the API first and want to scale to bulk later, your code keeps working with minimal changes.

Includes:

• Property core: type, year, size, bed/bath counts, lot dimensions
• Construction details: roof material, roof install year, foundation type, heating type, basement type
• Location: latitude, longitude, country, province, city, community, postal code
• Valuation: assessment value, assessment year, estimate sale price, estimate list price
• Risk scores: flood safety score, fire safety score, and others where available
• Demographics: census-derived neighbourhood attributes

Optional add-ons: listings history (list prices, days on market), permits, AVM time series.

Picking a plan

If you are still unsure, start on the API. The $99 a month minimum covers enough volume to validate that the data shape works for your project. If you find yourself making more than a few hundred thousand calls a month for read-heavy workloads, that is the moment to ask about a bulk dataset.

You can read the API docs at the quick start guide. For dataset pricing, the datasets page on the site lays it out. Either way, the same property data backs both products.