●Change8

0.25.0

Breaking Changes
📦 unstructuredView on GitHub →
⚠ 1 breaking✨ 2 features🐛 7 fixes⚡ 1 deprecations🔧 9 symbols

Summary

Enhanced JSON and NDJSON partitioning to handle arbitrary payloads and improved table chunk rehydration. Deprecated prefix-sniffing helpers for file type detection.

⚠️ Breaking Changes

  • NDJSON files where the first record exceeds 1 MiB will now be classified as JSON and cause `partition()` to raise a `ValueError`. Previously, these would be correctly identified as ND To fix this, call `partition_ndjson()` directly if you encounter this specific scenario.

Migration Steps

  1. If you encounter NDJSON files where the first record exceeds 1 MiB and `partition()` raises a `ValueError`, switch to calling `partition_ndjson()` directly for those specific files.

✨ New Features

  • `partition_json()` and `partition_ndjson()` now handle arbitrary JSON and NDJSON payloads, including bare objects, arrays of records, NDJSON lines, and scalars, by converting them to `Text` elements containing pretty-printed Previously, only serialized Unstructured output was handled, and other payloads would raise errors.
  • Serialized `TableChunk` elements are now correctly rehydrated by `elements_from_dicts()`, `partition_json()`, and `partition_ndjson()`. This completes the table-reconstruction feature, allowing `reconstruct_table_from_chunks()` to receive deserialized chunks.

🐛 Bug Fixes

  • The schema pre-gates in `partition()` have been removed, allowing for more flexible JSON and NDJSON handling.
  • A compact single-line JSON object will now be detected as `FileType.JSON` instead of ND The disambiguation logic examines up to the first 1 MiB of the file.
  • Malformed input to `partition_json()` and `partition_ndjson()` will continue to raise `ValueError`. Empty or whitespace-only documents will yield no elements.
  • If an element-shaped payload cannot be rehydrated (e.g., due to corrupt metadata), `ValueError` will be raised with the underlying error chained.
  • Arrays or NDJSON files containing a mix of element-shaped and arbitrary items will now be partitioned entirely as arbitrary JSON, without silently dropping arbitrary items.
  • An empty JSON object will now yield a single `Text` element containing `{}`. An empty JSON array will yield no elements.
  • A one-record serialized-element file (a single object, not an array) routed through `partition()`/`detect_filetype()` will now emit pretty-printed `Text` with alphabetized keys instead of rehydrating. Rehydration is now only applied to arrays.

Affected Symbols

partition_json()partition_ndjson()partition()detect_filetype()elements_from_dicts()reconstruct_table_from_chunks()is_json_processable()is_ndjson_processable()FileType.JSON

⚡ Deprecations

  • `is_json_processable()` and `is_ndjson_processable()` are deprecated and will be removed in a future release. Partitioning and file-type detection no longer use these prefix-sniffing helpers, but they will continue to work with a `DeprecationWarning` for downstream callers.