Rubric20-Semantic Evaluation Report

Strong field completion with comprehensive content in all major D4D sections. Primary gaps in structured metadata fields (version, publisher, citation) that exist in text but not as structured elements, and hierarchical/governance fields

Core fields present: id (DOI), title, description, keywords (25), license_and_use_terms, page, creators (15), purposes (3), instances (4), funders, subpopulations (7), collection_mechanisms (4), acquisition_methods (4), preprocessing_strategies (5), cleaning_strategies (3), intended_uses (6), discouraged_uses (3), distribution_formats (5), maintainers, updates, external_resources (12). Missing: version, publisher, citation, conforms_to, format, encoding, variables, is_tabular, resources, parent_datasets, related_datasets, confidentiality_level, regulatory_restrictions

Approximately 85% field completion - excellent coverage of motivation, composition, collection, preprocessing, uses, distribution, and maintenance sections

Excellent narrative field length throughout - all major descriptions exceed 200 characters with rich detail

description: 1,383 characters (23 lines); purposes[0].description: 430 chars; purposes[1].description: 386 chars; purposes[2].description: 339 chars; tasks descriptions: 200-400 chars each; collection_mechanisms descriptions: 300-600 chars each

All narrative fields substantially exceed minimum length requirement

Exceptional keyword diversity covering all major dataset dimensions

keywords: 25 unique keywords spanning methodologies (Cell Maps, AP-MS, SEC-MS, Immunofluorescence, Mass Spectrometry, Perturb-Seq), cell lines (MDA-MB-468, KOLF2.1J, iPSC), disease contexts (Breast Cancer), technologies (CRISPR Perturbation, Deep Learning, Visible Neural Networks), and standards (FAIR Principles, RO-Crate, FAIRSCAPE)

Well above threshold with comprehensive coverage of methods, biological systems, diseases, and standards

Highly multimodal dataset with 5 distinct distribution platforms and 4+ data modalities

distribution_formats: 5 formats - (1) RO-Crate packages with provenance, (2) Mass spec data in MassIVE, (3) Sequence data in NCBI SRA, (4) Hierarchical cell maps in NDEx, (5) Archives in UVA Dataverse. Multiple data types: imaging (IF confocal microscopy), proteomics (AP-MS, SEC-MS), genomics (CRISPR screens, single-cell RNA-seq), network data (hierarchical DAGs)

Excellent file type variety representing multimodal integration of imaging, proteomics, and genomics data

Instance counts and sample dimensions well-documented; file sizes not provided

instances: 4 documented (2 cell lines, 100 chromatin regulators, 100 metabolic enzymes); subpopulations: 7 conditions; subsets describe: 563 proteins imaged (IF), 17 genes tagged for AP-MS, 72/100 chromatin modifiers in SEC-MS, 11,739 genes in CRISPR atlas, >1,000 complexes detected; no bytes field for file sizes

Pass - dimensional metadata present through instance and subset descriptions

Excellent identifier coverage with multiple DOIs and valid RRIDs; hosting platforms described but not in publisher field

id: https://doi.org/10.18130/V3/DXWOS5 (primary DOI); Additional DOIs: 10.18130/V3/B35XWX (V1.4), 10.18130/V3/F3TD5R (V2.1), 10.1101/2024.05.21.589311 (bioRxiv); RRIDs: RRID:CVCL_0419 (MDA-MB-468), RRID:CVCL_B5P3 (KOLF2.1J); Hosting platforms mentioned: UVA Dataverse, MassIVE, NCBI SRA/BioProject, NDEx; no publisher field

Pass - comprehensive persistent identifier coverage; publisher field missing but platforms documented

Comprehensive funding documentation with grant numbers, amounts, dates, and complete creator affiliations

funders: NIH Common Fund Bridge2AI Program with grants 1OT2OD032742-01 and 5U54HG012513-02, Opportunity Number OTA-21-008, FY 2025 funding amounts ($5,289,382), project dates (Sep 2022 - Aug 2026), Frederick Thomas Fund; creators: 15 creators with roles (Contact PI, Co-Investigator) and affiliations (UCSD, SFU, UVA, UAB, UCSF, Stanford, UMontreal, Yale, UT Austin)

Exemplary funding transparency with complete grant details and creator attribution

Ethics oversight well-documented with named reviewers; de-identification described; missing structured fields for DPIA, privacy protections, and re-identification risk (though less critical for cell line data)

human_subject_research: involves_human_subjects=false, Ethics Module (Vardit Ravitsky, Jean-Christophe Bélisle-Pipon), Data Access Committee (Jillian Parker), Bridge2AI Ethics Working Group, de-identification status described ('cannot be matched to human subject'), special_populations (MDA-MB-468: 51-yo black female, KOLF2.1J: healthy male Northern European); sensitive_elements: de-identified cell lines; Missing: is_deidentified field, data_protection_impacts, participant_privacy, reidentification_risk, informed_consent, participant_compensation

Good ethics documentation appropriate for cell line data; could enhance with structured privacy/DPIA fields

Strong license documentation with governance oversight mentioned; lacks structured fields for regulatory compliance, confidentiality classification, and governance contact

license_and_use_terms: Comprehensive CC BY-NC-SA 4.0 with attribution requirements, non-commercial use only, commercial licensing available through UCSD/Stanford/UCSF, Data Access Committee oversight, must cite bioRxiv publication and data DOI; Missing: regulatory_restrictions, confidentiality_level, hipaa_compliant, other_compliance, governance_committee_contact (though Data Access Committee mentioned in text)

License well-specified; governance present but not in structured fields; missing multi-jurisdiction compliance metadata

Excellent standards adoption and integration capability via MuSIC pipeline and FAIRSCAPE; lacks structured conformance declarations

Standards mentioned: FAIR principles, RO-Crate (Research Object Crate), Schema.org, EVI Evidence Graph Ontology, Gene Ontology, Reactome, PDB, AlphaFold, JSON-Schema, JSON-LD; Integration: MuSIC pipeline integrates multimodal data (IF imaging, AP-MS, SEC-MS, CRISPR screens); FAIRSCAPE framework for AI-readiness; Missing: format, encoding, conforms_to, conforms_to_schema fields (standards described in text but not structured)

Strong interoperability with multiple standards and integration framework; would benefit from structured conforms_to field

Software tools comprehensively described throughout documentation with processing details; lacks structured software_and_tools field and annotation quality metrics

preprocessing_strategies: 5 strategies with tools (node2vec for PPI, Human Protein Atlas deep learning model for imaging, contrastive deep learning, Cytoscape for community detection, GO/Reactome/LLM for annotation); cleaning_strategies: 3 strategies (FAIRSCAPE-CLI for packaging, data standard mapping, integrative modeling pipeline); Software mentioned throughout: MuSIC pipeline, FAIRSCAPE, Cytoscape, HPA model, node2vec; Missing: software_and_tools field, annotation_analyses, machine_annotation_tools, imputation_protocols fields

Strong tool documentation in text form; would benefit from structured software_and_tools field with versions

Comprehensive collection protocols with technical details and lab attributions; lacks structured fields for collectors, timeframes, and raw data sources

collection_mechanisms: 4 detailed mechanisms (IF spatial proteomics with automated pipetting robot, AP-MS with endogenous tagging, SEC-MS for proteome-wide mapping, CRISPR screens with lentiviral library); acquisition_methods: 4 detailed methods (confocal microscopy 4-channel, mass spectrometry, single-cell RNA-seq with 10x Genomics 3'HT, MuSIC pipeline); Collectors implied: Lundberg Lab (Stanford), Krogan Lab (UCSF), Mali Lab (UCSD); Missing: data_collectors field, collection_timeframes field, raw_data_sources field (though source cell lines documented)

Excellent protocol documentation; would benefit from structured collector and timeframe fields

Strong version tracking and comprehensive sustainability plan; lacks structured version and version_access fields

updates: Quarterly release plan through Nov 2026, version history (v0.5 alpha, V1.4 beta March 2025 doi:10.18130/V3/B35XWX, V2.1 beta June 2025 doi:10.18130/V3/F3TD5R), change descriptions (perturb-seq, SEC-MS, IF images, RGB corrections), long-term preservation plan; retention_limit: LibraData (UVA Dataverse) with institutional commitment, NIH data sharing compliance; maintainers: CM4AI Consortium with 8 institutions; Sustainability: persistent identifiers (DOI/ARK), domain repository (Dataverse), institutional funding; Missing: version field, version_access field (though DOIs provide access)

Excellent sustainability with long-term preservation and update schedule; would benefit from version field

Publications linked with DOIs and citation required in license terms; lacks structured citation field with formatted string

external_resources: bioRxiv publication (Clark T, et al. Cell Maps for AI, doi:10.1101/2024.05.21.589311), Perturbation Cell Atlas publication (Nourreddine S, et al, doi:10.1101/2024.11.03.621734, PMCID: PMC11580897); license_and_use_terms requires citation of bioRxiv article and data DOI; Missing: citation field with formatted citation string, citation instructions not structured

Good publication linkage; missing structured citation field (mandatory for Bridge2AI)

Cell line origins documented with demographic context; not primary human subjects but diversity represented; lacks structured missing data documentation

instances: 2 human-derived cell lines (MDA-MB-468 from 51-yo black female, KOLF2.1J from healthy male Northern European); subpopulations: 7 conditions (3 MDA-MB-468 treatments, 4 KOLF2.1J differentiation states); human_subject_research.special_populations documents demographic origins; Missing: vulnerable_populations field, is_subpopulation field, missing_data_documentation field

Appropriate for cell line data with demographic context; not direct human subjects

Excellent findability with multiple persistent identifiers across different systems

id: https://doi.org/10.18130/V3/DXWOS5; page: https://www.cm4ai.org; Additional DOIs for versions: 10.18130/V3/B35XWX, 10.18130/V3/F3TD5R; ARK identifiers mentioned in FAIRSCAPE framework; RRIDs for cell lines

Pass - comprehensive persistent identifier coverage

Access mechanisms well-documented via multiple platforms; access tier implied through governance but not explicitly stated

distribution_formats: 5 platforms with access_urls (Dataverse DOIs, MassIVE, NCBI SRA/BioProject, NDEx, cm4ai.org); license_and_use_terms: CC BY-NC-SA with Data Access Committee oversight; Access tier implied: registration/DUA likely required given Data Access Committee mention; Missing: download_url field, explicit access tier classification

Good access documentation; would benefit from explicit access tier and download_url field

Excellent use guidance with clear intended and discouraged uses; social impact implied through purposes and gaps but not formal RAI-aligned analysis

license_and_use_terms: CC BY-NC-SA with clear terms; intended_uses: 6 uses (AI/ML training, VNN development, genotype-phenotype mapping, drug response prediction, disease mechanisms, AI-readiness model); discouraged_uses: 3 uses (clinical decision-making without validation, use during incomplete release, analysis without domain expertise); addressing_gaps describes transformative potential for AI in genomics; Missing: future_use_impacts field with formal social impact analysis and mitigation strategies

Strong reusability guidance; would benefit from structured future_use_impacts field

Excellent provenance via FAIRSCAPE with full entity-activity-agent tracking; lacks structured was_derived_from and parent_datasets fields but provenance comprehensively described

Provenance: FAIRSCAPE framework creates RO-Crate packages with datasets, metadata, provenance graphs (W3C PROV-O compatible), software; EVI Evidence Graph Ontology for provenance entailments; end-to-end provenance tracking from source cell lines (ATCC, HipSci) through MuSIC pipeline to hierarchical cell maps; updates: comprehensive update plan with version history; Quality: QC procedures documented for imaging and mass spec; Missing: was_derived_from field, parent_datasets field, missing_data_documentation field, is_data_split field (though provenance described)

Strong provenance graph representation through FAIRSCAPE RO-Crate system with W3C PROV-O alignment

Missing explicit bias documentation; dataset has inherent selection and population biases that should be documented using RAI-aligned taxonomy

No known_biases field present; No future_use_impacts field with fairness analysis; Potential biases not explicitly documented: selection bias (specific cell lines only - MDA-MB-468 and KOLF2.1J), population bias (limited diversity in cell line demographics - one black female donor, one Northern European male donor), representation bias (cancer and iPSC contexts only)

Major gap - no bias documentation despite clear selection and population biases in cell line choices