Course
data-engineering-zoomcamp
Question
After running the dlt pipeline, some fields from the API response (e.g. rate_code, mta_tax) are missing in the destination table.
Answer
dlt infers column types from actual data in the current load.
If a column contains only NULL values in that load, dlt cannot infer its type and the column will not be materialized.
To fix:
Provide explicit type hints using the columns argument in the @dlt.resource decorator, or
Ensure at least one non-null value exists for that column during ingestion.
Checklist
Course
data-engineering-zoomcamp
Question
After running the dlt pipeline, some fields from the API response (e.g. rate_code, mta_tax) are missing in the destination table.
Answer
dlt infers column types from actual data in the current load.
If a column contains only NULL values in that load, dlt cannot infer its type and the column will not be materialized.
To fix:
Provide explicit type hints using the columns argument in the @dlt.resource decorator, or
Ensure at least one non-null value exists for that column during ingestion.
Checklist