## What's Changed * Put upper bound on torchdata dependency, also fix some typing issues @ivirshup in https://github.com/chanzuckerberg/cellxgene-census/pull/1325 * Bump tiledbsoma pin to >=1.15.3 by @ivirshup in https://github.com/chanzuckerberg/cellxgene-census/pull/1336 * Update pin on tiledb-vector-search to ~=0.11 by @mlin in https://github.com/chanzuckerberg/cellxgene-census/pull/1349 * Numpy 2 support @ivirshup in https://github.com/chanzuckerberg/cellxgene-census/pull/1343 * Add warning about tiledbsoma library update to docsite by @ivirshup in https://github.com/chanzuckerberg/cellxgene-census/pull/1333 * [docs] Spatial documentation by @MaximilianLombardo in https://github.com/chanzuckerberg/cellxgene-census/pull/1334 * Add Spatial Census Tutorial by @MaximilianLombardo in https://github.com/chanzuckerberg/cellxgene-census/pull/1348 * Add 'spatial' extra install by @ivirshup in https://github.com/chanzuckerberg/cellxgene-census/pull/1346 * `cellxgene_census.experimental.ml` modules moved into deprecated status, and will be removed in a future release. @mlin https://github.com/chanzuckerberg/cellxgene-census/pull/1381 * `cellxgene_census.experimental.ml.pytorch` (data loader for PyTorch) is superseded by [TileDB-SOMA-ML](https://github.com/single-cell-data/TileDB-SOMA-ML); [tutorial notebook](https://chanzuckerberg.github.io/cellxgene-census/notebooks/experimental/pytorch.html) revised accordingly. * `cellxgene_census.experimental.ml.huggingface` (Hugging Face Dataset builder and Geneformer tokenizer) will migrate to tools/models/geneformer in the git repository (not part of the PyPI package) * Add support for getting data from census_spatial_sequencing to `get_anndata`, `get_obs`, `get_var` by @ivirshup in https://github.com/chanzuckerberg/cellxgene-census/pull/1383 ## New Contributors * @jakeyheath made their first contribution in https://github.com/chanzuckerberg/cellxgene-census/pull/1335 * @alexlokshin-czi made their first contribution in https://github.com/chanzuckerberg/cellxgene-census/pull/1351 **Full Changelog**: https://github.com/chanzuckerberg/cellxgene-census/compare/v1.16.2...v1.17.0

## What's Changed * [python] Fix vector search import by @ivirshup in https://github.com/chanzuckerberg/cellxgene-census/pull/1287 * [python] Pin against tiledbsoma 1.14.1 by @ivirshup in https://github.com/chanzuckerberg/cellxgene-census/pull/1290 * chore(docs): Add banner and newsletter signup (SCE-29) by @kaloster in https://github.com/chanzuckerberg/cellxgene-census/pull/1286 * [Python] Correct docs on default behaviour for DataLoader no longer including soma_joinid by @ivirshup in https://github.com/chanzuckerberg/cellxgene-census/pull/1292 ## New Contributors * @kaloster made their first contribution in https://github.com/chanzuckerberg/cellxgene-census/pull/1286 **Full Changelog**: https://github.com/chanzuckerberg/cellxgene-census/compare/v1.16.1...v1.16.2

## What's Changed * [python] Fix vector search import by @ivirshup in https://github.com/chanzuckerberg/cellxgene-census/pull/1287 * [python] Pin against tiledbsoma 1.14.1 by @ivirshup in https://github.com/chanzuckerberg/cellxgene-census/pull/1290 * [Python] Correct docs on default behaviour for DataLoader no longer including soma_joinid by @ivirshup in https://github.com/chanzuckerberg/cellxgene-census/pull/1292 ## New Contributors * @kaloster made their first contribution in https://github.com/chanzuckerberg/cellxgene-census/pull/1286 **Full Changelog**: https://github.com/chanzuckerberg/cellxgene-census/compare/v1.16.1...v1.16.2

## What's Changed * [python] Fix `predict_obs_metadata` when number of neighbors isn't `10` by @ivirshup in https://github.com/chanzuckerberg/cellxgene-census/pull/1277 **Full Changelog**: https://github.com/chanzuckerberg/cellxgene-census/compare/v1.16.0...v1.16.1

## What's Changed * [python] Embeddings search experimental API by @mlin in https://github.com/chanzuckerberg/cellxgene-census/pull/1164 * [python] Add comp_bio_embedding_search notebook by @mlin @pablo-gar in https://github.com/chanzuckerberg/cellxgene-census/pull/1241 https://github.com/chanzuckerberg/cellxgene-census/pull/1270 * Add FAQ on access from behind a proxy by @ivirshup in https://github.com/chanzuckerberg/cellxgene-census/pull/1266 * [docs] Update landing page with latest LTS data and API release. "Adds Release notes" link by @pablo-gar in https://github.com/chanzuckerberg/cellxgene-census/pull/1239 * [python] Update minimum versions of dependencies + python versions following [SPEC-0](https://scientific-python.org/specs/spec-0000/) by @ivirshup in https://github.com/chanzuckerberg/cellxgene-census/pull/1189 * [python] Bump minimum tiledbsoma version to 1.12.3 by @ivirshup in https://github.com/chanzuckerberg/cellxgene-census/pull/1235 * [docs] Fix model names in metrics article by @pablo-gar in https://github.com/chanzuckerberg/cellxgene-census/pull/1248 * [docs] Fix date on `cellxgene_census_docsite_data_release_info.md` by @pablo-gar in https://github.com/chanzuckerberg/cellxgene-census/pull/1259 * Add Boroni collection to blocklist by @ivirshup in https://github.com/chanzuckerberg/cellxgene-census/pull/1262 * feat: Update navbar to include Differential Expression by @tihuan in https://github.com/chanzuckerberg/cellxgene-census/pull/1264 **Full Changelog**: https://github.com/chanzuckerberg/cellxgene-census/compare/v1.15.0...v1.16.0

## What's Changed * [python] Shuffle multiple SOMA chunks by @atolopko-czi in https://github.com/chanzuckerberg/cellxgene-census/pull/1103 * [python] Relax pin: `tiledbsoma~=1.11.4` by @ryan-williams in https://github.com/chanzuckerberg/cellxgene-census/pull/1187 * [pytorch] Set shuffle as default in pytorch, use new algorithm by @ebezzi in https://github.com/chanzuckerberg/cellxgene-census/pull/1188 * [python] Remove pytorch pin by @ebezzi in https://github.com/chanzuckerberg/cellxgene-census/pull/1190 * [builder] Vector indexing pipeline for Census embeddings by @mlin in https://github.com/chanzuckerberg/cellxgene-census/pull/1122 * [builder] Add collection_doi_label to the datasets dataframe by @ebezzi in https://github.com/chanzuckerberg/cellxgene-census/pull/1200 * [python] Require scikit-misc < 0.4 by @ebezzi in https://github.com/chanzuckerberg/cellxgene-census/pull/1213 * [builder] Upgrade to CELLxGENE schema 5.1 by @Bento007 in https://github.com/chanzuckerberg/cellxgene-census/pull/1192 * [docs] Update schema doc for version 2.1.0 by @ebezzi in https://github.com/chanzuckerberg/cellxgene-census/pull/1211 * [python] Update scvi pipeline for the June LTS training by @ebezzi in https://github.com/chanzuckerberg/cellxgene-census/pull/1173 * [python] Check census version for get_all_available_embeddings by @ivirshup in https://github.com/chanzuckerberg/cellxgene-census/pull/1207 * [python] Move to {obs/var} specific arguments for get_anndata by @ivirshup in https://github.com/chanzuckerberg/cellxgene-census/pull/1149 * [python] Stop including `"do_not_delete"` key in result of `get_census_version_directory` by @ivirshup in https://github.com/chanzuckerberg/cellxgene-census/pull/1215 * [python] Add package specific user-agent by @ivirshup in https://github.com/chanzuckerberg/cellxgene-census/pull/1193 * [python] Update docs for `get_obs`, `get_var` by @ivirshup in https://github.com/chanzuckerberg/cellxgene-census/pull/1170 * [python] Geneformer updates for July 2024 LTS by @mlin in https://github.com/chanzuckerberg/cellxgene-census/pull/961 * [python] Support custom obs encoders by @ebezzi in https://github.com/chanzuckerberg/cellxgene-census/pull/1191 * [python] Fix ExperimentDataPipe length by @ebezzi in https://github.com/chanzuckerberg/cellxgene-census/pull/1221 * [python] dataloader optimization (picking up 1169) by @ivirshup in https://github.com/chanzuckerberg/cellxgene-census/pull/1224 * [docs] Update census data release page with upcoming LTS information by @pablo-gar in https://github.com/chanzuckerberg/cellxgene-census/pull/1210 * [docs] Add PyTorch loaders article release by @pablo-gar in https://github.com/chanzuckerberg/cellxgene-census/pull/1214 * [python] Geneformer CLS embeddings by @mlin in https://github.com/chanzuckerberg/cellxgene-census/pull/1225 * [docs] Update the Logistic Regression pytorch notebook by @ebezzi in https://github.com/chanzuckerberg/cellxgene-census/pull/1230 * [docs] Add metrics article for LTS 2024-12-15 by @pablo-gar in https://github.com/chanzuckerberg/cellxgene-census/pull/1233 ## New Contributors * @ryan-williams made their first contribution in https://github.com/chanzuckerberg/cellxgene-census/pull/1187 **Full Changelog**: https://github.com/chanzuckerberg/cellxgene-census/compare/v1.14.1...v1.15.0

## Major changes - Upgrade to `tiledbsoma` 1.11.3 - Add a [cellxgene_census.get_obs](https://github.com/chanzuckerberg/cellxgene-census/blob/main/api/python/cellxgene_census/src/cellxgene_census/_get_anndata.py#L175) utility function that can be used to return an obs dataframe in Pandas format more easily. ## What's Changed * [builder] Update list of accepted assays and bump schema to 2.0.1 by @ebezzi in https://github.com/chanzuckerberg/cellxgene-census/pull/1117 * [docs] update scvi notebook to use latest model by @pablo-gar in https://github.com/chanzuckerberg/cellxgene-census/pull/1137 * [docs] Update geneformer notebook to latest API changes by @pablo-gar in https://github.com/chanzuckerberg/cellxgene-census/pull/1138 * [docs] Add development setup notes for Census API by @prathapsridharan in https://github.com/chanzuckerberg/cellxgene-census/pull/1111 * [python] Add get_obs by @ivirshup in https://github.com/chanzuckerberg/cellxgene-census/pull/1151 * [python] share census objects in tests for faster test time by @ivirshup in https://github.com/chanzuckerberg/cellxgene-census/pull/1156 * [builder] Bump requests from 2.31.0 to 2.32.0 in /tools/cellxgene_census_builder by @dependabot in https://github.com/chanzuckerberg/cellxgene-census/pull/1154 * [builder] Add ClientResponseError to the builder managed exceptions by @ebezzi in https://github.com/chanzuckerberg/cellxgene-census/pull/1126 * [python] [R] Bump tiledbsoma to 1.11.3 ## New Contributors * @ivirshup made their first contribution in https://github.com/chanzuckerberg/cellxgene-census/pull/1151 **Full Changelog**: https://github.com/chanzuckerberg/cellxgene-census/compare/v1.13.1...v1.14.0

## New embeddings API Census embeddings can now accessed using a new, simplified API. Check the notebooks for [collaboration](https://github.com/chanzuckerberg/cellxgene-census/blob/main/api/python/notebooks/api_demo/census_access_maintained_embeddings.ipynb) and [hosted](https://github.com/chanzuckerberg/cellxgene-census/blob/main/api/python/notebooks/api_demo/census_embedding.ipynb) models for more information. ## obs columns are now categorical instead of strings Starting from the `2024-04-01` Census build, a subset of the columns in the `obs` dataframe are now categorical instead of strings. For **Python users**, note that Pandas will encode these columns as `pandas.Categorical` for which some downstream operations may need to be adapted. See [this link](https://pandas.pydata.org/docs/user_guide/categorical.html#operations) for more details. In particular: > Series methods like Series.value_counts() will use all categories, even if some categories are not present in the data and > DataFrame methods like sum, groupby, pivot, value_counts also show “unused” categories when observed=False, which is the default. For **R users**, note that these columns will be encoded as `factor` and similarly downstream operations may need to be adapted. See [this link](https://r4ds.had.co.nz/factors.html) more details. For **Python and R users** interfacing with `arrow`, these columns will be encoded as `dictionary`, see more details for R in [this link](https://arrow.apache.org/docs/r/reference/dictionary.html) and Python [this link](https://arrow.apache.org/docs/python/generated/pyarrow.dictionary.html). ## Additions * [builder] enable Arrow Dictionary feature flag by @bkmartinjr in https://github.com/chanzuckerberg/cellxgene-census/pull/1064 * [python] New embeddings API by @ebezzi in https://github.com/chanzuckerberg/cellxgene-census/pull/1023 * [docs] New embeddings API notebooks by @ebezzi in https://github.com/chanzuckerberg/cellxgene-census/pull/1070 **Full Changelog**: https://github.com/chanzuckerberg/cellxgene-census/compare/v1.12.0...v1.13.0