# Interleaved Semantic Evaluation

Project: CHORUS · Method: claudecode_agent_core
YAML: `data/d4d_concatenated/claudecode_agent_core/CHORUS_d4d_core.yaml`
R10 JSON: `data/evaluation_llm/rubric10_semantic/concatenated/CHORUS_claudecode_agent_core_evaluation.json`
R20 JSON: `data/evaluation_llm/rubric20_semantic/concatenated/CHORUS_claudecode_agent_core_evaluation.json`
Model: `claude-sonnet-4-5-20250929`
Rubric10 (semantic)
34/50 (68.0%)
Rubric20 (semantic)
58.0/84 (69.0%)
Consistency checks (R10/R20)
43 pass · 2 fail · 9 warn
Mapped feedback / fields
159 across 48 fields
R10 sub-element R20 question Semantic issue

### Strengths

- Exceptional structural completeness with all mandatory fields populated and comprehensive narrative content (1,347+ char description, 28 keywords, 4 detailed purposes)
- Outstanding funding documentation with complete NIH grant details (OT2OD032701, opportunity number, study section, fiscal details, timeline, $5.88M)
- Excellent multi-modal file format diversity (5 distribution types: OMOP, WFDB, DICOM, OHNLP/TXT, EDF+/Persyst) with recognized industry standards
- Comprehensive ethics documentation covering IRB approvals (14 sites), vulnerable populations (critically ill, PICU, NICU), deidentification methods, and informed consent framework
- Very good collection protocol clarity with detailed acquisition methods (5 modalities), mechanisms (4 processes), collector details (14 acquisition centers), and timeframes
- Clear governance and access documentation with controlled access license, DUA requirements, regulatory compliance (HIPAA, 45 CFR 46), and contact information
- Strong findability with multiple persistent URLs (project website, GitHub organization with 28 repos, NIH RePORTER, publication DOI, Bridge2AI program links)
- All grant numbers, DOI, regulatory citations, and URLs correctly formatted and semantically valid

### Weaknesses

- D4D-core schema limitations: lacks bytes field for structured file sizes, may lack DOI/RRID, conforms_to, version, errata, release_notes, citation, participant_compensation, participant_privacy fields
- Software tools mentioned by name (OHNLP, OHDSI, privacy_scan_tool, DeGauss) but lacking version numbers and repository URLs
- Limited demographic detail in instances/subpopulations despite diversity claims in purposes - no age ranges, sex, race/ethnicity distributions provided
- Minimal data integrity/provenance documentation - update narrative provided but no structured change logs, version timestamps, or formal version history
- Limited cross-platform data repository interlinking - no PhysioNet link despite WFDB format use, no data commons or other repository registrations
- No formal schema conformance references (conforms_to/conforms_to_schema fields) despite using multiple standards (OMOP, DICOM, WFDB, OHNLP)

## Field-by-field

id

    id: https://chorus4ai.org/

⚠ low R20 · schema_limitation
issue Core schema may lack DOI, conforms_to, conforms_to_schema fields
fields id, distribution_formats
fix Verify if D4D-core schema includes DOI and schema conformance fields. Standards mentioned narratively (OMOP, DICOM, WFDB, OHNLP) but no formal conforms_to references.
✗ 0/1 R10 1.Dataset Discovery and Identification Persistent Identifier (DOI, RRID, or URI)
evidence id: https://chorus4ai.org/ (URI present but not DOI or RRID)
quality D4D-core schema lacks doi and rrid fields; only id (URI) present
semantic URI is valid project website but not a persistent identifier from a recognized registrar (DataCite, Crossref, RRID registry)
✗ 0/1 R10 10.Cross-Platform and Community Integration Citation and DOI for Cross-referencing
evidence No citation or doi fields in D4D-core schema; external_resources reference peer-reviewed publication (Neurocritical Care journal, doi: 10.1007/s12028-024-02007) documenting methodology but not dataset DOI; id field contains URI (https://chorus4ai.org/) but not DOI
quality D4D-core schema lacks citation and doi fields; publication DOI available for methodology paper (10.1007/s12028-024-02007) but no dataset DOI for direct citation; id field uses URI rather than persistent identifier
semantic Lack of dataset DOI limits cross-referencing and citation tracking; methodology paper DOI provides some citability but not direct dataset citation
✗ 0/1 R10 4.Ethical Use and Privacy Safeguards Vulnerable Populations and Compensation Documented
evidence at_risk_populations: Critically ill patients (ICU, PICU, NICU) vulnerable due to acute illness; pediatric and neonatal patients particularly vulnerable; patients with diverse social determinants of health; privacy protections include de-id + controlled access + DUA; community ethics focus groups engaged; no participant_compensation field in D4D-core
quality At-risk populations clearly identified (critically ill ICU/PICU/NICU patients) with appropriate privacy protections; D4D-core schema lacks participant_compensation field; retrospective waiver-of-consent framework means no compensation discussion applicable
semantic Vulnerable population protections appropriate for critically ill patient data; retrospective data collection with waiver of consent means participant compensation not applicable
✓ 1/1 R10 5.Data Composition and Structure Data Quality Issues and Anomalies Documented
evidence known_limitations with 2 entries: (1) Dataset actively growing, not yet complete (45K current vs 100K+ target, EEG extraction in process, imaging de-id in process, notes stored locally); (2) Controlled access restricts open research (DUA required, enclave imposes computational constraints); known_biases with 2 entries: (1) Academic medical center population bias (may not represent community/rural settings); (2) Retrospective data collection bias (documentation practices vary by site/modality/time); missing_data_documentation: Incomplete modality coverage across sites (EEG/imaging in process, notes local-only)
quality Comprehensive quality documentation: known limitations address dataset completion status and access constraints; known biases address selection bias (academic centers) and documentation bias (retrospective collection); missing data documentation addresses modality coverage gaps
semantic Quality issues and biases semantically appropriate for multi-institutional retrospective clinical data collection; transparency about ongoing expansion and site-to-site variability demonstrates data quality awareness
✗ 0/1 R10 6.Data Provenance and Version Tracking Change Descriptions and Errata Provided
evidence No errata field in D4D-core schema; updates describe ongoing expansion (45K → 100K+ admissions) and processing status (EEG extraction in process, imaging de-id in process) but no structured change log or errata documentation
quality D4D-core schema lacks errata field; updates section describes ongoing expansion but not version-to-version changes or corrections
semantic Change documentation present in narrative form (ongoing collection through Nov 2026) but not structured errata or version release notes
✓ 1/1 R10 9.Dataset Evaluation and Limitations Disclosure Known Limitations Documented
evidence known_limitations with 2 entries: (1) Dataset actively growing, not yet complete (Aug 2025: 14 hospitals, 45K+ unique admissions, 50K released vs target 100K+; EEG extraction in process, imaging de-id in process, notes stored locally with only tokens in enclave; cohort coverage varies by modality; ongoing collection through Nov 2026); (2) Controlled access restricts open research (DUA required, institutional email registration, secure enclave imposes computational/logistical constraints)
quality Explicit known_limitations section with clear documentation of dataset completeness status, ongoing expansion timeline, modality-specific processing status, and access constraints; provides actionable information for dataset users about current vs. planned coverage
semantic Limitations semantically appropriate for large-scale ongoing data generation project: transparency about growth trajectory, modality processing status, and access model constraints
✓ 1/1 R10 9.Dataset Evaluation and Limitations Disclosure Data Anomalies and Quality Issues Noted
evidence missing_data_documentation: Incomplete modality coverage across sites (not all modalities available from all 14 centers; EEG extraction and imaging de-id in process as of Aug 2025; clinical notes stored locally with only tokens in enclave; cohort coverage varies by modality during ongoing collection); cleaning_strategies mention cross-site data quality checks via CHoRUSReports; known_limitations mention documentation completeness/accuracy varying by site/modality/time
quality Data quality issues documented: incomplete modality coverage across sites, ongoing processing for EEG/imaging, federated note storage (only tokens centralized), cross-site variability in documentation practices; cleaning_strategies describe quality assurance via CHoRUSReports characterization
semantic Quality issues semantically appropriate for large-scale multi-institutional data aggregation: site-to-site variability in data availability and completeness expected, ongoing processing status provides transparency about dataset maturity
✓ 1/1 R10 9.Dataset Evaluation and Limitations Disclosure Sensitive Content and Warnings Provided
evidence sensitive_elements with 4 entries: (1) Protected health information - clinical EHR data (complete EHRs including demographics, diagnoses, procedures, medications, nursing documentation, clinical notes; subject to HIPAA and institutional privacy requirements; de-identified before release via OMOP transformation and privacy scanning); (2) Physiological monitoring and EEG waveform data (continuous waveforms revealing sensitive health conditions and treatment responses; WFDB/EDF+/Persyst formats with controlled access); (3) Medical imaging data (diagnostic imaging in DICOM with embedded patient information; de-id in process); (4) Social Determinants of Health data (geographic information, SDOH data; privacy-preserving transformations applied)
quality Comprehensive sensitive_elements documentation identifying all sensitive data types (PHI in EHR, physiological states in waveforms, diagnostic information in imaging, geographic/SDOH data) with privacy protections described for each; no content_warnings field in D4D-core but sensitive_elements serves similar purpose
semantic Sensitive content identification semantically appropriate for clinical dataset: PHI, physiological data, diagnostic imaging, and SDOH all require privacy protections; descriptions link sensitivity to applicable regulations (HIPAA) and mitigation strategies (de-identification, controlled access)
5/5 R20 Q1 (Structural Completeness) Field Completeness
level≥90% fields populated
evidence id: https://chorus4ai.org/, title: Patient-Focused Collaborative Hospital Repository Uniting Standards (CHoRUS) for Equitable AI, description: 420+ chars comprehensive, keywords: 28 keywords, license: Controlled Access - Data Use Agreement Required (OT2OD032701)
quality All mandatory fields present with comprehensive, detailed content. Description provides specific project scope, data types, and current dataset size.
correctness id_format: valid_url; title_appropriateness: descriptive and accurate; license_clarity: controlled access clearly specified with grant reference
consistency id_matches_page: True; keywords_align_with_description: True; license_matches_access_requirements: True
3/5 R20 Q11 (Technical Documentation) Tool and Software Transparency
level Strategies listed but limited tool details
evidence preprocessing_strategies: 5 strategies (OMOP transformation, OHNLP tokenization, WFDB standardization, DICOM de-id, re-id limitation). cleaning_strategies: 2 strategies (multi-center harmonization, schema compliance). labeling_strategies: 1 strategy (visualization and annotation environment). Software tools mentioned: privacy_scan_tool, DeGauss (UF-Geocoding), OHDSI tool stack, OHNLP toolkit, but no version numbers or URLs.
quality Good coverage of strategies across preprocessing, cleaning, and labeling. Software tools mentioned by name but lacking version numbers, GitHub links, or detailed documentation references. Core schema may not have dedicated software_and_tools field.
correctness tool_names: OHNLP, OHDSI, DeGauss, privacy_scan_tool are plausible tool names; preprocessing_methods: OMOP transformation, tokenization, WFDB conversion appropriate for described data types
consistency preprocessing_aligns_with_formats: OMOP preprocessing matches OMOP distribution format; tools_match_strategies: OHNLP toolkit matches clinical note tokenization strategy; github_references: mentions chorus-mapping and chorus_waveform repos but not full URLs
1/1 R20 Q6 (Metadata Quality & Content) Dataset Identification Metadata
level Pass - persistent URL
evidence id: https://chorus4ai.org/, page: https://chorus4ai.org/. No DOI or RRID present in core schema.
quality Persistent URL identifier present but no formal DOI. Core schema may not include DOI field - this is a schema limitation rather than content quality issue.
correctness url_format: valid HTTPS URL; url_domain: chorus4ai.org is plausible domain for CHoRUS project; doi_format: not_present; rrid_format: not_present
consistency id_matches_page: True; note: DOI may be schema limitation in D4D-core
name

    name: CHoRUS

no field-level feedback matched
title

    title: Patient-Focused Collaborative Hospital Repository Uniting Standards (CHoRUS) for Equitable AI

✓ 1/1 R10 1.Dataset Discovery and Identification Dataset Title and Description Completeness
evidence title: Patient-Focused Collaborative Hospital Repository Uniting Standards (CHoRUS) for Equitable AI; description: 1,156 chars with specific details (100K+ patients, 14 centers, 1.6B rows, 23TB waveforms, multi-modal data types)
quality Comprehensive description with quantitative details, project scope, data modalities, standardization approaches, and current status (Aug 2025)
semantic Description semantically rich and appropriate for multi-center critical care AI dataset; provides actionable information about scale, diversity, and data types
5/5 R20 Q1 (Structural Completeness) Field Completeness
level≥90% fields populated
evidence id: https://chorus4ai.org/, title: Patient-Focused Collaborative Hospital Repository Uniting Standards (CHoRUS) for Equitable AI, description: 420+ chars comprehensive, keywords: 28 keywords, license: Controlled Access - Data Use Agreement Required (OT2OD032701)
quality All mandatory fields present with comprehensive, detailed content. Description provides specific project scope, data types, and current dataset size.
correctness id_format: valid_url; title_appropriateness: descriptive and accurate; license_clarity: controlled access clearly specified with grant reference
consistency id_matches_page: True; keywords_align_with_description: True; license_matches_access_requirements: True
description

    description: 'CHoRUS for Equitable AI is a Bridge2AI data generation project developing the most diverse,
      high-resolution, ethically sourced, AI-ready critical care dataset to answer the grand challenge of
      improving recovery from acute illness. The project spans 20 academic centers (14 data acquisition centers)
      and is building a publicly available dataset targeting over 100,000 critically ill patients with multi-modal
      data including structured EHR, waveform telemetry, medical imaging, EEG, and clinical notes. All structured
      data is standardized to the OMOP Common Data Model with additional formats (DICOM, WFDB, OHNLP tokenization)
      and comprehensive metadata schemas. Patient-focused efforts determine ethical and legal approaches to
      manage privacy and bias while accounting for Social Determinants of Health. A visualization and annotation
      environment labels data with targets important for prediction. The project emphasizes skills and workforce
      development for a next generation of diverse academic and community AI scientists through training programs
      and partnerships with AIM-AHEAD. As of August 2025, the dataset covers 14 different hospitals with over
      45,000 unique admissions and includes 50,000 patient admissions from ICU, PICU, and NICU, 1.6 billion
      rows of EHR OMOP data, 7,642 admissions with radiology data, and 23 TB of waveform data.

      '

✓ 1/1 R10 1.Dataset Discovery and Identification Dataset Title and Description Completeness
evidence title: Patient-Focused Collaborative Hospital Repository Uniting Standards (CHoRUS) for Equitable AI; description: 1,156 chars with specific details (100K+ patients, 14 centers, 1.6B rows, 23TB waveforms, multi-modal data types)
quality Comprehensive description with quantitative details, project scope, data modalities, standardization approaches, and current status (Aug 2025)
semantic Description semantically rich and appropriate for multi-center critical care AI dataset; provides actionable information about scale, diversity, and data types
✗ 0/1 R10 2.Dataset Access and Retrieval Download URL or Platform Link Available
evidence distributions[*].path all point to https://chorus4ai.org/ (no direct download URLs); description mentions 'secure enclave' and 'controlled access'
quality D4D-core schema lacks download_url field; distribution paths point to project website rather than direct download links; access through secure enclave requires DUA approval
semantic Controlled access model means no public download URLs; users must register and receive access instructions after approval
5/5 R20 Q1 (Structural Completeness) Field Completeness
level≥90% fields populated
evidence id: https://chorus4ai.org/, title: Patient-Focused Collaborative Hospital Repository Uniting Standards (CHoRUS) for Equitable AI, description: 420+ chars comprehensive, keywords: 28 keywords, license: Controlled Access - Data Use Agreement Required (OT2OD032701)
quality All mandatory fields present with comprehensive, detailed content. Description provides specific project scope, data types, and current dataset size.
correctness id_format: valid_url; title_appropriateness: descriptive and accurate; license_clarity: controlled access clearly specified with grant reference
consistency id_matches_page: True; keywords_align_with_description: True; license_matches_access_requirements: True
5/5 R20 Q2 (Structural Completeness) Entry Length Adequacy
level>200 chars
evidence description: 1,347 chars, purposes: 4 entries averaging 350+ chars each, tasks: 5 entries with 150-250 chars each
quality Exceptionally detailed narrative content across all descriptive fields. Provides specific numbers, technical details, and comprehensive context.
correctness description_accuracy: specific metrics (100,000 patients, 14 centers, 23TB waveform data); purposes_clarity: clear, measurable objectives aligned with critical care AI
consistency description_matches_purposes: True; quantitative_claims_consistent: True
page

    page: https://chorus4ai.org/

✓ 1/1 R10 1.Dataset Discovery and Identification Landing Page and Resources (page, hierarchical resources)
evidence page: https://chorus4ai.org/; external_resources with 8 entries including project website, GitHub organization (28 repos), SOP documentation, NIH RePORTER, Bridge2AI program, AIM-AHEAD training, OHDSI/OMOP community, peer-reviewed publication (doi: 10.1007/s12028-024-02007)
quality Active landing page plus comprehensive external resources with documentation, code repositories, training materials, and published literature
semantic Landing page and resources provide multiple access points for dataset understanding, technical documentation, and community engagement
✓ 1/1 R10 10.Cross-Platform and Community Integration Outreach Materials and Documentation Links
evidence external_resources with 8 entries: CHoRUS project website, GitHub organization (28 repos including software, documentation, SOPs, tooling at https://github.com/chorus-ai), Chorus_SOP documentation site with interactive workflow diagrams (https://github.com/chorus-ai/Chorus_SOP), NIH RePORTER project details, Bridge2AI program page, AIM-AHEAD training program (https://aim-ahead.net/), OHDSI community (https://www.ohdsi.org/), peer-reviewed publication (https://doi.org/10.1007/s12028-024-02007); existing_uses mention AIM-AHEAD training with Jupyter Notebooks and OHDSI/OMOP workshops
quality Extensive outreach and documentation: project website, GitHub organization with 28 repositories, standard operating procedure documentation with interactive diagrams, training program partnerships (AIM-AHEAD with Jupyter Notebooks and workshops), published methodology paper; documentation covers technical implementation (SOPs, GitHub repos), educational materials (training program, workshops), and scientific context (publication)
semantic Outreach materials semantically comprehensive: technical documentation (GitHub, SOPs) for developers, training materials (AIM-AHEAD, Jupyter Notebooks) for users, scientific publications for researchers, community links (OHDSI) for ecosystem integration
1/1 R20 Q16 (FAIRness & Accessibility) Findability (Persistent Links)
level Pass - persistent URLs present
evidence page: https://chorus4ai.org/, external_resources: 8 resources with URLs including GitHub (https://github.com/chorus-ai), NIH RePORTER (https://reporter.nih.gov/project-details/10472824), Bridge2AI (https://bridge2ai.org/chorus), publication DOI (https://doi.org/10.1007/s12028-024-02007)
quality Multiple persistent URLs provided including project website, GitHub organization, federal grant database, and publication DOI. Strong findability through multiple access points.
correctness url_validity: all URLs use HTTPS protocol; domain_plausibility: chorus4ai.org, github.com, nih.gov, bridge2ai.org, doi.org are all valid domains; nih_reporter_id: 10472824 is plausible NIH project ID format
consistency page_matches_id: id and page both use https://chorus4ai.org/; external_urls_align_with_project: GitHub org chorus-ai matches project name CHoRUS
1/1 R20 Q6 (Metadata Quality & Content) Dataset Identification Metadata
level Pass - persistent URL
evidence id: https://chorus4ai.org/, page: https://chorus4ai.org/. No DOI or RRID present in core schema.
quality Persistent URL identifier present but no formal DOI. Core schema may not include DOI field - this is a schema limitation rather than content quality issue.
correctness url_format: valid HTTPS URL; url_domain: chorus4ai.org is plausible domain for CHoRUS project; doi_format: not_present; rrid_format: not_present
consistency id_matches_page: True; note: DOI may be schema limitation in D4D-core
language

    language: en

✓ 1/1 R10 3.Data Reuse and Interoperability Data Formats Are Standardized (encoding, format)
evidence distribution_formats: OMOP Common Data Model (structured EHR), WFDB (waveform telemetry), DICOM (medical imaging), OHNLP tokenized (clinical notes), EDF+/Persyst (EEG); distributions[3] specifies format: TXT, media_type: text/plain; language: en
quality All data modalities use community-standard formats: OMOP CDM (OHDSI standard), WFDB (PhysioNet standard), DICOM (medical imaging standard), OHNLP (open source NLP), EDF+ (European Data Format for EEG); language specified as English
semantic Formats are internationally recognized standards enabling interoperability; OMOP enables OHDSI tool stack, DICOM enables medical imaging workflows, WFDB enables PhysioNet tools
license

    license: Controlled Access - Data Use Agreement Required (OT2OD032701)

✓ 1/1 R10 3.Data Reuse and Interoperability License Terms Allow Reuse
evidence license: Controlled Access - Data Use Agreement Required (OT2OD032701); license_and_use_terms: CHoRUS Controlled Access License with DUA requiring institutional email registration, signed agreement, review/approval process
quality Clear license terms allowing reuse for research purposes under controlled access model; DUA process documented with contact information (dbold@emory.edu, jared.houghtaling@tuftsmedicine.org)
semantic Controlled access with DUA is appropriate reuse model for de-identified clinical data; balances data sharing with privacy protection
✓ 1/1 R10 7.Scientific Motivation and Funding Transparency Grant IDs or Award Numbers Present
evidence funders: grant OT2OD032701 (project number 1OT2OD032701-01); license mentions 'OT2OD032701'; Total funding in 2022: $5,880,300 (all direct costs); Project dates: September 1, 2022 to November 30, 2026; Award notice date: September 1, 2022
quality Grant number present in multiple formats (OT2OD032701, 1OT2OD032701-01) with funding amount ($5.88M in 2022), project dates (Sept 2022 - Nov 2026), and award notice date (Sept 1, 2022); grant number follows NIH OT2 (Other Transaction Agreement) format
semantic Grant number format valid for NIH: OT2 (mechanism) + OD (NIH Office of the Director) + 032701 (serial number); suffix '-01' likely indicates first year/segment; funding details provide financial transparency
5/5 R20 Q1 (Structural Completeness) Field Completeness
level≥90% fields populated
evidence id: https://chorus4ai.org/, title: Patient-Focused Collaborative Hospital Repository Uniting Standards (CHoRUS) for Equitable AI, description: 420+ chars comprehensive, keywords: 28 keywords, license: Controlled Access - Data Use Agreement Required (OT2OD032701)
quality All mandatory fields present with comprehensive, detailed content. Description provides specific project scope, data types, and current dataset size.
correctness id_format: valid_url; title_appropriateness: descriptive and accurate; license_clarity: controlled access clearly specified with grant reference
consistency id_matches_page: True; keywords_align_with_description: True; license_matches_access_requirements: True
4/5 R20 Q18 (FAIRness & Accessibility) Reusability (License Clarity)
level License explicitly defines reuse terms with contact details
evidence license_and_use_terms: controlled access license requiring institutional email + signed agreement, prohibits re-identification and redistribution, research purposes only. ip_restrictions: NIH grant terms (OT2OD032701), clinical data ownership with contributing institutions. prohibited_uses: re-identification prohibited, redistribution outside DUA prohibited.
quality License clearly defined with explicit reuse restrictions: research purposes only, no re-identification, no redistribution. Permitted uses implied through intended_uses (AI/ML development, validation, health equity research, education). Minor deduction for no explicit permitted use enumeration in license field itself.
correctness controlled_access_appropriate: correct for sensitive de-identified clinical data; prohibition_specificity: re-identification and redistribution prohibitions are standard for health data
consistency license_aligns_with_intended_uses: research purposes matches intended_uses (AI/ML development, education); license_restrictions_match_prohibited_uses: DUA prohibitions match prohibited_uses section; ip_ownership_clear: institutional ownership with NIH grant terms
publisher

    publisher: CHoRUS Consortium / Massachusetts General Hospital

✓ 1/1 R10 10.Cross-Platform and Community Integration Dataset Published on a Recognized Platform
evidence publisher: CHoRUS Consortium / Massachusetts General Hospital; distributions available at https://chorus4ai.org/ via secure enclave with controlled access; external_resources reference NIH RePORTER (https://reporter.nih.gov/project-details/10472824) and Bridge2AI program (https://bridge2ai.org/chorus)
quality Dataset published through CHoRUS Consortium infrastructure (secure enclave at chorus4ai.org) with NIH Bridge2AI program affiliation; not on traditional data repository platform (PhysioNet, Dataverse, Zenodo) but accessible through project-specific infrastructure with NIH program backing
semantic Publisher semantically appropriate: CHoRUS Consortium is multi-institutional collaboration led by MGH under NIH Bridge2AI program; controlled access via project-specific secure enclave reflects sensitive clinical data requirements
is_tabular

    is_tabular: false

✗ 0/1 R10 3.Data Reuse and Interoperability Variable Metadata with Identifiers Defined
evidence No variables field in D4D-core schema; is_tabular: false indicates non-tabular multi-modal dataset
quality D4D-core schema lacks variables field; dataset is multi-modal (structured EHR in OMOP, waveforms in WFDB, imaging in DICOM, notes in OHNLP) rather than single tabular format requiring variable-level metadata
semantic Variable-level metadata not applicable for multi-modal dataset; OMOP CDM tables have their own schema documentation (referenced in external_resources)
✓ 1/1 R10 5.Data Composition and Structure Variable-Level Metadata and Tabular Flag
evidence is_tabular: false; distribution_formats describe multi-modal data (OMOP CDM structured EHR, WFDB waveforms, DICOM imaging, OHNLP tokenized notes, EDF+/Persyst EEG); no variables field in D4D-core
quality Tabular flag correctly set to false for multi-modal dataset; D4D-core lacks variables field but distribution_formats and preprocessing_strategies describe each modality's structure; OMOP CDM schema metadata published externally (referenced in external_resources)
semantic is_tabular=false semantically correct for dataset with multiple distinct formats (structured EHR, waveforms, imaging, text, EEG); variable metadata delegated to modality-specific schemas (OMOP CDM tables, DICOM tags, WFDB headers)
keywords

    keywords:
    - CHoRUS
    - Bridge2AI
    - critical care
    - acute illness
    - AI-ready dataset
    - OMOP Common Data Model
    - electronic health records
    - EHR
    - waveform telemetry
    - medical imaging
    - DICOM
    - EEG
    - clinical notes
    - OHNLP
    - health equity
    - social determinants of health
    - federated access
    - multi-modal data
    - high-resolution data
    - ethical AI
    - trustworthy AI
    - workforce development
    - data standardization
    - privacy preservation
    - bias mitigation
    - ICU
    - intensive care

✓ 1/1 R10 1.Dataset Discovery and Identification Keywords or Tags for Searchability
evidence keywords: [CHoRUS, Bridge2AI, critical care, acute illness, AI-ready dataset, OMOP Common Data Model, EHR, waveform telemetry, medical imaging, DICOM, EEG, clinical notes, OHNLP, health equity, social determinants of health, federated access, multi-modal data, high-resolution data, ethical AI, trustworthy AI, workforce development, data standardization, privacy preservation, bias mitigation, ICU, intensive care] (27 keywords)
quality Extensive keywords covering domain (critical care, ICU), methods (OMOP, DICOM, WFDB, OHNLP), concepts (health equity, ethical AI, bias mitigation), and project identity (CHoRUS, Bridge2AI)
semantic Keywords semantically appropriate and comprehensive for discovery; cover data modalities, standards, research domains, and ethical considerations
5/5 R20 Q1 (Structural Completeness) Field Completeness
level≥90% fields populated
evidence id: https://chorus4ai.org/, title: Patient-Focused Collaborative Hospital Repository Uniting Standards (CHoRUS) for Equitable AI, description: 420+ chars comprehensive, keywords: 28 keywords, license: Controlled Access - Data Use Agreement Required (OT2OD032701)
quality All mandatory fields present with comprehensive, detailed content. Description provides specific project scope, data types, and current dataset size.
correctness id_format: valid_url; title_appropriateness: descriptive and accurate; license_clarity: controlled access clearly specified with grant reference
consistency id_matches_page: True; keywords_align_with_description: True; license_matches_access_requirements: True
5/5 R20 Q3 (Structural Completeness) Keyword Diversity
level≥8 keywords
evidence keywords: 28 unique keywords including CHoRUS, Bridge2AI, critical care, acute illness, AI-ready dataset, OMOP Common Data Model, electronic health records, EHR, waveform telemetry, medical imaging, DICOM, EEG, clinical notes, OHNLP, health equity, social determinants of health, federated access, multi-modal data, high-resolution data, ethical AI, trustworthy AI, workforce development, data standardization, privacy preservation, bias mitigation, ICU, intensive care
quality Excellent keyword diversity covering data types (EHR, waveform, imaging), standards (OMOP, DICOM, OHNLP), domains (critical care, ICU), and AI themes (equity, ethics, trustworthy AI).
correctness keyword_relevance: all keywords semantically appropriate for critical care dataset; technical_terms_accuracy: OMOP, DICOM, WFDB, OHNLP correctly used
consistency keywords_match_content: True; no_contradictory_keywords: True
distributions

    distributions:
    - id: chorus:dist:1
      description: 'Structured electronic health record data in OMOP Common Data Model format (proprietary
        clinical data standard). Contains 1.6 billion rows. Available in secure enclave with controlled access
        at https://chorus4ai.org/. Published OMOP schema metadata available.

        '
      path: https://chorus4ai.org/
    - id: chorus:dist:2
      description: 'Waveform telemetry data (23 TB) in WFDB (WaveForm DataBase) format following extended
        PhysioNet schema (binary waveform data format). Available in secure enclave with controlled access
        at https://chorus4ai.org/.

        '
      path: https://chorus4ai.org/
    - id: chorus:dist:3
      description: 'Medical imaging data (7,642 admissions with radiology; 1,000 images currently available)
        in DICOM format (Digital Imaging and Communications in Medicine) with comprehensive metadata. De-identification
        in process for larger cohort. Planned controlled access at https://chorus4ai.org/.

        '
      path: https://chorus4ai.org/
    - id: chorus:dist:4
      description: 'Clinical notes distributed as OHNLP-tokenized plain text following open source OHNLP schema.
        Stored locally at contributing sites (except tokens). Controlled access planned at https://chorus4ai.org/.

        '
      format: TXT
      media_type: text/plain
      path: https://chorus4ai.org/
    - id: chorus:dist:5
      description: 'EEG waveform data in EDF+ (European Data Format) and Persyst formats following open source
        schemas (binary physiological signal formats). Extraction in process, planned for controlled access
        at https://chorus4ai.org/.

        '
      path: https://chorus4ai.org/

⚠ low R20 · schema_limitation
issue D4D-core schema lacks bytes field for structured file size metadata
fields distributions
fix Size information provided narratively (23TB waveform, 1.6B rows) but not in structured metadata. This is a core schema limitation, not a content quality issue.
✓ 1/1 R10 10.Cross-Platform and Community Integration Dataset Published on a Recognized Platform
evidence publisher: CHoRUS Consortium / Massachusetts General Hospital; distributions available at https://chorus4ai.org/ via secure enclave with controlled access; external_resources reference NIH RePORTER (https://reporter.nih.gov/project-details/10472824) and Bridge2AI program (https://bridge2ai.org/chorus)
quality Dataset published through CHoRUS Consortium infrastructure (secure enclave at chorus4ai.org) with NIH Bridge2AI program affiliation; not on traditional data repository platform (PhysioNet, Dataverse, Zenodo) but accessible through project-specific infrastructure with NIH program backing
semantic Publisher semantically appropriate: CHoRUS Consortium is multi-institutional collaboration led by MGH under NIH Bridge2AI program; controlled access via project-specific secure enclave reflects sensitive clinical data requirements
✗ 0/1 R10 2.Dataset Access and Retrieval Download URL or Platform Link Available
evidence distributions[*].path all point to https://chorus4ai.org/ (no direct download URLs); description mentions 'secure enclave' and 'controlled access'
quality D4D-core schema lacks download_url field; distribution paths point to project website rather than direct download links; access through secure enclave requires DUA approval
semantic Controlled access model means no public download URLs; users must register and receive access instructions after approval
✓ 1/1 R10 2.Dataset Access and Retrieval Distribution Formats and File Types Specified
evidence distribution_formats with 5 entries: OMOP CDM (structured EHR), WFDB (waveform telemetry, 23TB), DICOM (medical imaging), OHNLP tokenized (clinical notes, TXT/text/plain), EDF+/Persyst (EEG); distributions[3] specifies format: TXT, media_type: text/plain for clinical notes
quality Comprehensive format documentation across all data modalities with specific standards (OMOP, WFDB, DICOM, OHNLP, EDF+/Persyst); clinical notes include media_type
semantic Formats semantically appropriate for clinical data types: OMOP for structured EHR, WFDB for physiological waveforms, DICOM for imaging, OHNLP for NLP-ready text, EDF+ for EEG
✓ 1/1 R10 3.Data Reuse and Interoperability Data Formats Are Standardized (encoding, format)
evidence distribution_formats: OMOP Common Data Model (structured EHR), WFDB (waveform telemetry), DICOM (medical imaging), OHNLP tokenized (clinical notes), EDF+/Persyst (EEG); distributions[3] specifies format: TXT, media_type: text/plain; language: en
quality All data modalities use community-standard formats: OMOP CDM (OHDSI standard), WFDB (PhysioNet standard), DICOM (medical imaging standard), OHNLP (open source NLP), EDF+ (European Data Format for EEG); language specified as English
semantic Formats are internationally recognized standards enabling interoperability; OMOP enables OHDSI tool stack, DICOM enables medical imaging workflows, WFDB enables PhysioNet tools
1/5 R20 Q15 (Technical Documentation) Human Subject Representation
level General human data with limited demographic details
evidence instances: critically ill patients in ICU/PICU/NICU settings, 14 hospitals, 45,000 unique admissions current, 50,000 released, 100,000+ target. subpopulations: by hospital (14 hospitals) and ICU type (ICU, PICU, NICU). No detailed demographics (age ranges, sex, race/ethnicity distributions).
quality Describes patient population and setting (critically ill, ICU/PICU/NICU) with quantitative details (50K admissions, 14 sites) but lacks detailed demographic breakdowns. Mentions diversity and Social Determinants of Health in purposes but not in instances/subpopulations.
correctness population_counts: 45K unique, 50K released, 100K+ target are plausible and internally consistent; site_count: 14 hospitals consistent across fields; population_type: critically ill ICU/PICU/NICU appropriate for critical care dataset
consistency instance_counts_consistent: 45K unique admissions and 50K total admissions (some patients may have multiple admissions); subpopulation_coverage: ICU/PICU/NICU subpopulations match instance description; diversity_claims_not_quantified: purposes mention diversity but no demographic breakdowns provided
4/5 R20 Q17 (FAIRness & Accessibility) Accessibility (Access Mechanism)
level Fully defined access path
evidence license_and_use_terms: institutional email registration required, signed licensing agreement, access request contacts (dbold@emory.edu, jared.houghtaling@tuftsmedicine.org). distribution_formats describe controlled access via secure enclave for each data type. distributions specify access at https://chorus4ai.org/ with controlled access requirements.
quality Very clear access mechanism: institutional email registration → signed licensing agreement → secure enclave access. Contact emails provided. Specific requirements documented. Minor deduction for no download vs. enclave-only clarification.
correctness email_format: dbold@emory.edu and jared.houghtaling@tuftsmedicine.org are valid email formats; institutional_email_requirement: appropriate for controlled access to sensitive health data; secure_enclave_mention: Azure-based infrastructure mentioned elsewhere
consistency access_requirements_match_license: institutional email and DUA consistent with controlled access license; all_distributions_same_mechanism: all 5 distributions use same controlled access pattern; deidentified_but_controlled: correctly maintains controlled access despite de-identification
5/5 R20 Q4 (Structural Completeness) File Enumeration and Type Variety
level>3 file types
evidence distributions: 5 distributions with formats including OMOP Common Data Model, WFDB waveform format, DICOM imaging, OHNLP tokenized text (TXT/text/plain), EDF+ and Persyst EEG formats
quality Exceptional file type diversity representing multi-modal critical care data: structured EHR (OMOP), waveforms (WFDB), imaging (DICOM), text (OHNLP/TXT), and EEG (EDF+/Persyst).
correctness format_standards: all formats are recognized industry standards (OMOP, DICOM, WFDB, EDF+); media_type_accuracy: text/plain correctly specified for TXT files
consistency distribution_formats_match_descriptions: True; format_variety_matches_multimodal_claims: True
0/1 R20 Q5 (Structural Completeness) Data File Size Availability
level Fail - no bytes field
evidence instances: 1 instance entry describing patient population. No bytes field present in distributions. Size mentioned narratively in descriptions (23TB waveform, 1.6 billion rows OMOP, 50,000 admissions)
quality D4D-core schema lacks bytes field. Quantitative size information embedded in description text but not structured metadata. Core schema limitation.
correctness instance_count_accuracy: 50,000 admissions mentioned, 100,000+ target; narrative_sizes: 23TB waveform, 1.6B rows OMOP mentioned in descriptions
consistency size_claims_across_fields: consistent 50K current, 100K+ target across multiple fields; note: D4D-core schema does not include bytes or structured size fields
purposes

    purposes:
    - id: chorus:purpose:1
      name: Improve recovery from acute illness
      description: 'Answer the grand challenge of improving recovery from acute illness by developing high-resolution
        multi-center datasets as a critical first step towards actionable and trustworthy AI in critical care.
        Address the urgent need for infrastructure to support artificial intelligence and machine learning
        (AI/ML) in critical care settings.

        '
    - id: chorus:purpose:2
      name: Create AI-ready critical care dataset
      description: 'Develop a publicly available, AI-ready critical care dataset from more than 100,000 critically
        ill patients while ensuring methods promote privacy, accountability, and clinical benefit. Generate
        the most diverse, high-resolution, ethically sourced dataset for AI/ML applications in acute and critical
        care, expanding AI and Machine Learning to improve recovery from acute illness.

        '
    - id: chorus:purpose:3
      name: Establish data standards and tools
      description: 'Unify standards to harmonize multi-modal EHR, waveform, imaging, and text data. Develop
        software and tooling to interact with and extract insight from clinical data in diverse formats. Create
        validated semantic mappings for connecting clinical data in various source formats to international
        standards (OMOP Common Data Model, DICOM, WFDB, OHNLP).

        '
    - id: chorus:purpose:4
      name: Promote diversity and health equity
      description: 'Ensure comprehensive sets of patient conditions and clinical treatment strategies with
        appropriate contextual factors such as geographic distance to nearest hospital and Social Determinants
        of Health. Develop the skills and workforce for a next generation of diverse academic and community
        AI scientists through comprehensive training and education programs in partnership with AIM-AHEAD.

        '

⚠ medium R20 · completeness
issue Demographic diversity claims not quantified in instances/subpopulations
fields instances, subpopulations, purposes
fix Purposes mention diversity and Social Determinants of Health but no demographic breakdowns (age, sex, race/ethnicity) in instances. Consider adding demographic subpopulations.
✓ 1/1 R10 5.Data Composition and Structure Data Topics or Conditions Represented
evidence instances: critically ill patients requiring acute or critical care in ICU/PICU/NICU; purposes address 'acute illness', 'critical care illness'; tasks include 'characterize acute and critical care illness patterns', 'predict complications among patients with acute or critical illness', 'measure treatment response'; addressing_gaps: lack of diverse multi-center critical care datasets
quality Clear topic focus on acute and critical care illness, ICU patient populations, complications, treatment response; purposes and tasks provide specific research questions (illness characterization, complication prediction, treatment measurement)
semantic Topics semantically coherent around critical care medicine, acute illness, ICU outcomes; multi-modal data (EHR, waveforms, imaging, notes, EEG) appropriate for comprehensive critical illness characterization
✓ 1/1 R10 7.Scientific Motivation and Funding Transparency Motivation or Purpose for Dataset Creation
evidence purposes with 4 entries: (1) Improve recovery from acute illness by developing high-resolution multi-center datasets as critical first step towards actionable/trustworthy AI in critical care; (2) Create AI-ready critical care dataset from 100K+ critically ill patients while ensuring privacy, accountability, clinical benefit; (3) Establish data standards and tools to unify multi-modal EHR/waveform/imaging/text data; (4) Promote diversity and health equity through comprehensive patient conditions, contextual factors (SDOH), and workforce development via AIM-AHEAD
quality Clear scientific motivation across four dimensions: clinical impact (improve acute illness recovery), dataset development (AI-ready multi-center data), technical infrastructure (standards/tools), societal impact (diversity/equity/workforce); addressing_gaps with 4 entries provides additional context
semantic Motivation semantically rich and coherent: addresses clinical need (critical care AI), technical gaps (multi-center harmonization), ethical considerations (privacy, bias, equity), and workforce development
4/5 R20 Q18 (FAIRness & Accessibility) Reusability (License Clarity)
level License explicitly defines reuse terms with contact details
evidence license_and_use_terms: controlled access license requiring institutional email + signed agreement, prohibits re-identification and redistribution, research purposes only. ip_restrictions: NIH grant terms (OT2OD032701), clinical data ownership with contributing institutions. prohibited_uses: re-identification prohibited, redistribution outside DUA prohibited.
quality License clearly defined with explicit reuse restrictions: research purposes only, no re-identification, no redistribution. Permitted uses implied through intended_uses (AI/ML development, validation, health equity research, education). Minor deduction for no explicit permitted use enumeration in license field itself.
correctness controlled_access_appropriate: correct for sensitive de-identified clinical data; prohibition_specificity: re-identification and redistribution prohibitions are standard for health data
consistency license_aligns_with_intended_uses: research purposes matches intended_uses (AI/ML development, education); license_restrictions_match_prohibited_uses: DUA prohibitions match prohibited_uses section; ip_ownership_clear: institutional ownership with NIH grant terms
5/5 R20 Q2 (Structural Completeness) Entry Length Adequacy
level>200 chars
evidence description: 1,347 chars, purposes: 4 entries averaging 350+ chars each, tasks: 5 entries with 150-250 chars each
quality Exceptionally detailed narrative content across all descriptive fields. Provides specific numbers, technical details, and comprehensive context.
correctness description_accuracy: specific metrics (100,000 patients, 14 centers, 23TB waveform data); purposes_clarity: clear, measurable objectives aligned with critical care AI
consistency description_matches_purposes: True; quantitative_claims_consistent: True
tasks

    tasks:
    - id: chorus:task:1
      name: Characterize acute and critical care illness
      description: 'Generate data for ML/AI applications aimed at characterizing acute and critical care illness
        patterns, progression, and outcomes across diverse patient populations and hospital settings.

        '
    - id: chorus:task:2
      name: Predict complications in critically ill patients
      description: 'Enable prediction of complications among patients with acute or critical illness using
        multi-modal data including structured EHR, waveforms, imaging, and clinical notes.

        '
    - id: chorus:task:3
      name: Measure treatment response
      description: 'Support measurement and analysis of treatment response among critically ill patients through
        high-frequency documentation, medication administration records, and clinical outcomes data.

        '
    - id: chorus:task:4
      name: External validation for AI model marketplace adoption
      description: 'Provision a holdout test set accessible for model external validation to aid marketplace
        adoption of AI-developed models for implementation in acute and critical care settings.

        '
    - id: chorus:task:5
      name: Label data for prediction targets
      description: 'Utilize visualization and annotation environment to label data with targets important
        for prediction tasks in critical care AI applications.

        '

✓ 1/1 R10 5.Data Composition and Structure Data Topics or Conditions Represented
evidence instances: critically ill patients requiring acute or critical care in ICU/PICU/NICU; purposes address 'acute illness', 'critical care illness'; tasks include 'characterize acute and critical care illness patterns', 'predict complications among patients with acute or critical illness', 'measure treatment response'; addressing_gaps: lack of diverse multi-center critical care datasets
quality Clear topic focus on acute and critical care illness, ICU patient populations, complications, treatment response; purposes and tasks provide specific research questions (illness characterization, complication prediction, treatment measurement)
semantic Topics semantically coherent around critical care medicine, acute illness, ICU outcomes; multi-modal data (EHR, waveforms, imaging, notes, EEG) appropriate for comprehensive critical illness characterization
✓ 1/1 R10 7.Scientific Motivation and Funding Transparency Primary Research Objectives or Tasks
evidence tasks with 5 entries: (1) Characterize acute and critical care illness patterns/progression/outcomes; (2) Predict complications in critically ill patients using multi-modal data; (3) Measure treatment response through high-frequency documentation/medication records/outcomes; (4) External validation for AI model marketplace adoption; (5) Label data for prediction targets via visualization/annotation environment
quality Specific research objectives covering illness characterization, complication prediction, treatment response measurement, external validation, and data annotation; tasks align with purposes and provide actionable research questions
semantic Tasks semantically appropriate for critical care AI research: descriptive (characterization), predictive (complications), evaluative (treatment response), methodological (external validation, annotation)
5/5 R20 Q2 (Structural Completeness) Entry Length Adequacy
level>200 chars
evidence description: 1,347 chars, purposes: 4 entries averaging 350+ chars each, tasks: 5 entries with 150-250 chars each
quality Exceptionally detailed narrative content across all descriptive fields. Provides specific numbers, technical details, and comprehensive context.
correctness description_accuracy: specific metrics (100,000 patients, 14 centers, 23TB waveform data); purposes_clarity: clear, measurable objectives aligned with critical care AI
consistency description_matches_purposes: True; quantitative_claims_consistent: True
addressing_gaps

    addressing_gaps:
    - id: chorus:gap:1
      name: Lack of diverse multi-center critical care datasets
      description: 'Address the absence of large-scale, diverse, high-resolution multi-center datasets for
        critical care AI/ML by creating a dataset spanning 20 academic centers with 100,000+ critically ill
        patients and ensuring balanced, diverse cohorts through federated access and sampling methods across
        14 data acquisition centers.

        '
    - id: chorus:gap:2
      name: Insufficient data standardization in critical care
      description: 'Overcome the lack of unified standards in critical care data by harmonizing multi-modal
        EHR, waveform, imaging, and text data to OMOP Common Data Model and other international standards
        (DICOM, WFDB, OHNLP).

        '
    - id: chorus:gap:3
      name: Privacy and bias concerns in clinical AI
      description: 'Address ethical and legal challenges in AI through patient-focused efforts that determine
        approaches to manage privacy and bias while accounting for Social Determinants of Health and performing
        community-facing ethics focus groups to determine what data is appropriate for public sharing.

        '
    - id: chorus:gap:4
      name: Limited diversity in AI/ML workforce
      description: 'Bridge the gap in diverse AI/ML workforce through comprehensive educational approaches,
        training programs (including AIM-AHEAD partnership), and cultivation of expertise in lay and scientific
        communities to improve AI literacy and utilization.

        '

✓ 1/1 R10 5.Data Composition and Structure Data Topics or Conditions Represented
evidence instances: critically ill patients requiring acute or critical care in ICU/PICU/NICU; purposes address 'acute illness', 'critical care illness'; tasks include 'characterize acute and critical care illness patterns', 'predict complications among patients with acute or critical illness', 'measure treatment response'; addressing_gaps: lack of diverse multi-center critical care datasets
quality Clear topic focus on acute and critical care illness, ICU patient populations, complications, treatment response; purposes and tasks provide specific research questions (illness characterization, complication prediction, treatment measurement)
semantic Topics semantically coherent around critical care medicine, acute illness, ICU outcomes; multi-modal data (EHR, waveforms, imaging, notes, EEG) appropriate for comprehensive critical illness characterization
creators

    creators:
    - id: chorus:creator:1
      name: Eric S. Rosenthal
      description: Contact PI/Project Leader, Massachusetts General Hospital (MGH), Director of MGH Neurosciences
        ICU
    - id: chorus:creator:2
      name: Azra Bihorac
      description: Principal Investigator, University of Florida (UF)
    - id: chorus:creator:3
      name: Ashley Cordes
      description: Principal Investigator, CHoRUS Consortium
    - id: chorus:creator:4
      name: Gilles Clermont
      description: Principal Investigator, CHoRUS Consortium
    - id: chorus:creator:5
      name: Gari David Clifford
      description: Principal Investigator, CHoRUS Consortium
    - id: chorus:creator:6
      name: Barbara J. Evans
      description: Principal Investigator, CHoRUS Consortium
    - id: chorus:creator:7
      name: Xiao Hu
      description: Principal Investigator, CHoRUS Consortium
    - id: chorus:creator:8
      name: Rishikesan Kamaleswaran
      description: Principal Investigator, CHoRUS Consortium
    - id: chorus:creator:9
      name: Yulia A. Levites Strekalova
      description: Principal Investigator, University of Florida (UF)
    - id: chorus:creator:10
      name: Parisa Rashidi
      description: Principal Investigator, University of Florida (UF)
    - id: chorus:creator:11
      name: Cynthia Rudin
      description: Principal Investigator, CHoRUS Consortium
    - id: chorus:creator:12
      name: Ishan Canty Williams
      description: Principal Investigator, CHoRUS Consortium
    - id: chorus:creator:13
      name: Andrew Ewing Williams
      description: Principal Investigator, Tufts Medicine
    - id: chorus:creator:14
      name: Xiaoqian Jiang
      description: Program Lead, UT Health Science Center (UTHealth Houston)
    - id: chorus:creator:15
      name: Morteza Zabihi
      description: Lecturer (Machine Learning Basics), Massachusetts General Hospital
    - id: chorus:creator:16
      name: Zhenhong Hu
      description: Instructor (Python and Version Control), University of Florida
    - id: chorus:creator:17
      name: Debora Simmons
      description: Lecturer (Ethics of AI in Clinical Practice), UT Health Science Center
    - id: chorus:creator:18
      name: Aliyah Geer
      description: Workshop Lead (Data Schemas in Clinical Cloud), Massachusetts General Hospital
    - id: chorus:creator:19
      name: Ciera McCrary
      description: Program Manager, Massachusetts General Hospital (MGH)

✓ 1/1 R10 7.Scientific Motivation and Funding Transparency Creators and Acknowledgements Documented
evidence creators with 19 entries: Contact PI Eric S. Rosenthal (MGH, Director of MGH Neurosciences ICU), multiple PIs from UF, Tufts, and CHoRUS Consortium (Azra Bihorac, Ashley Cordes, Gilles Clermont, Gari Clifford, Barbara Evans, Xiao Hu, Rishikesan Kamaleswaran, Yulia Levites Strekalova, Parisa Rashidi, Cynthia Rudin, Ishan Williams, Andrew Williams), Program Lead Xiaoqian Jiang (UTHealth Houston), instructors/lecturers (Morteza Zabihi, Zhenhong Hu, Debora Simmons, Aliyah Geer), Program Manager Ciera McCrary (MGH)
quality Comprehensive creator list with roles (Contact PI, PI, Program Lead, Lecturer, Instructor, Workshop Lead, Program Manager) and institutional affiliations (MGH, UF, UTHealth, Tufts, CHoRUS Consortium); 19 named individuals provide accountability and expertise visibility
semantic Creator documentation reflects multi-institutional collaboration structure with leadership roles (Contact PI, PIs), technical expertise (Program Lead), training roles (lecturers/instructors), and program management
5/5 R20 Q7 (Metadata Quality & Content) Funding and Acknowledgements Completeness
level Funders with grants + creators with affiliations
evidence funders: NIH Common Fund Bridge2AI Program with grant OT2OD032701, project 1OT2OD032701-01, fiscal details ($5,880,300 in 2022), dates (Sept 1, 2022 to Nov 30, 2026). creators: 19 creators with institutional affiliations (MGH, UF, UTHealth, Tufts, etc.) and roles specified
quality Exceptional funding documentation with complete grant details including opportunity number (OTA-21-008), study section, fiscal year, funding amounts, project dates, award notice date, and assistance listing number. All creators have institutional affiliations.
correctness grant_number_format: OT2OD032701 follows NIH Other Transaction (OT) format correctly; grant_number_validation: 1OT2OD032701-01 matches NIH pattern [Type][Number]-[Sequence]; funding_amount_plausibility: $5.88M reasonable for Bridge2AI data generation project; date_validity: Sept 2022 to Nov 2026 is valid 4-year timeline with extension
consistency grant_numbers_consistent: OT2OD032701 and 1OT2OD032701-01 are same award; dates_align_with_project_timeline: True; creator_institutions_match_acquisition_centers: MGH, UF mentioned in both creators and acquisition centers
funders

    funders:
    - id: chorus:funder:1
      name: NIH Common Fund Bridge2AI Program
      description: 'Funded through National Institutes of Health grant OT2OD032701 (project number 1OT2OD032701-01),
        administered by NIH Office of the Director. Opportunity Number: OTA-21-008. Study Section: Data Coordination,
        Mapping, and Modeling (DCMM). Fiscal Year 2022. Total funding in 2022: $5,880,300 (all direct costs).
        Project dates: September 1, 2022 to November 30, 2026 (with approved no-cost extension). Award notice
        date: September 1, 2022. Assistance Listing Number: 93.310.

        '

⚠ low R10 · correctness
issue Grant number format '1OT2OD032701-01' follows NIH pattern but includes suffix '-01' which may be fiscal year/segment notation rather than core award number
fields funders
fix Confirm whether core award number is OT2OD032701 or 1OT2OD032701-01
✓ 1/1 R10 7.Scientific Motivation and Funding Transparency Funding Sources and Mechanisms Listed
evidence funders: NIH Common Fund Bridge2AI Program via National Institutes of Health grant OT2OD032701 (project number 1OT2OD032701-01), administered by NIH Office of the Director; Opportunity Number: OTA-21-008; Study Section: Data Coordination, Mapping, and Modeling (DCMM); Fiscal Year 2022; Assistance Listing Number: 93.310
quality Comprehensive funding details: agency (NIH Common Fund), program (Bridge2AI), grant number (OT2OD032701 / 1OT2OD032701-01), administering office (NIH Office of the Director), opportunity number (OTA-21-008), study section (DCMM), fiscal year (2022), assistance listing (93.310)
semantic Funding details semantically complete with grant number, program context, and administrative metadata; NIH Common Fund Bridge2AI program appropriate for large-scale AI-ready dataset generation
✓ 1/1 R10 7.Scientific Motivation and Funding Transparency Grant IDs or Award Numbers Present
evidence funders: grant OT2OD032701 (project number 1OT2OD032701-01); license mentions 'OT2OD032701'; Total funding in 2022: $5,880,300 (all direct costs); Project dates: September 1, 2022 to November 30, 2026; Award notice date: September 1, 2022
quality Grant number present in multiple formats (OT2OD032701, 1OT2OD032701-01) with funding amount ($5.88M in 2022), project dates (Sept 2022 - Nov 2026), and award notice date (Sept 1, 2022); grant number follows NIH OT2 (Other Transaction Agreement) format
semantic Grant number format valid for NIH: OT2 (mechanism) + OD (NIH Office of the Director) + 032701 (serial number); suffix '-01' likely indicates first year/segment; funding details provide financial transparency
5/5 R20 Q7 (Metadata Quality & Content) Funding and Acknowledgements Completeness
level Funders with grants + creators with affiliations
evidence funders: NIH Common Fund Bridge2AI Program with grant OT2OD032701, project 1OT2OD032701-01, fiscal details ($5,880,300 in 2022), dates (Sept 1, 2022 to Nov 30, 2026). creators: 19 creators with institutional affiliations (MGH, UF, UTHealth, Tufts, etc.) and roles specified
quality Exceptional funding documentation with complete grant details including opportunity number (OTA-21-008), study section, fiscal year, funding amounts, project dates, award notice date, and assistance listing number. All creators have institutional affiliations.
correctness grant_number_format: OT2OD032701 follows NIH Other Transaction (OT) format correctly; grant_number_validation: 1OT2OD032701-01 matches NIH pattern [Type][Number]-[Sequence]; funding_amount_plausibility: $5.88M reasonable for Bridge2AI data generation project; date_validity: Sept 2022 to Nov 2026 is valid 4-year timeline with extension
consistency grant_numbers_consistent: OT2OD032701 and 1OT2OD032701-01 are same award; dates_align_with_project_timeline: True; creator_institutions_match_acquisition_centers: MGH, UF mentioned in both creators and acquisition centers
instances

    instances:
    - id: chorus:instance:1
      name: Critically ill patients in ICU settings
      description: 'Individual critically ill patients requiring acute or critical care admitted to intensive
        care units (ICU), pediatric intensive care units (PICU), or neonatal intensive care units (NICU) across
        14 data acquisition centers. As of August 2025, the dataset covers 14 different hospitals with over
        45,000 unique admissions, with 50,000 patient admissions from ICU, PICU, and NICU currently released.
        Target enrollment exceeds 100,000 critically ill patients. Retrospective data collection from patients
        with acute or critical illness.

        '

⚠ medium R20 · completeness
issue Demographic diversity claims not quantified in instances/subpopulations
fields instances, subpopulations, purposes
fix Purposes mention diversity and Social Determinants of Health but no demographic breakdowns (age, sex, race/ethnicity) in instances. Consider adding demographic subpopulations.
✓ 1/1 R10 5.Data Composition and Structure Cohort or Subpopulations Characteristics Described
evidence subpopulations with 2 entries: (1) Patients distributed across 14 hospitals within CHoRUS network spanning 20 academic centers in US ensuring geographic/institutional diversity; (2) ICU, PICU, NICU patients with 50,000 admissions in current release; instances: 14 hospitals, 45K+ unique admissions (Aug 2025), 50K patient admissions (ICU/PICU/NICU), target 100K+ critically ill patients
quality Clear subpopulation descriptions with geographic diversity (14 hospitals, 20 academic centers), care setting diversity (ICU/PICU/NICU), and scale (50K current, 100K+ target); instances provide additional quantitative details
semantic Subpopulation characteristics semantically rich with specific institutional diversity, patient type diversity, and enrollment targets
✓ 1/1 R10 5.Data Composition and Structure Number of Instances or Samples Reported
evidence instances: 14 different hospitals, over 45,000 unique admissions (Aug 2025), 50,000 patient admissions from ICU/PICU/NICU currently released, target exceeds 100,000 critically ill patients; additional details: 1.6 billion rows of EHR OMOP data, 7,642 admissions with radiology data, 23 TB of waveform data, 1,000 images currently available
quality Comprehensive instance counts across multiple dimensions: patient admissions (50K current, 100K+ target), unique admissions (45K+), hospitals (14), OMOP rows (1.6B), waveform data (23TB), radiology admissions (7.6K), images (1K)
semantic Instance counts semantically appropriate for multi-modal dataset; provides granular quantification for each data modality
✓ 1/1 R10 5.Data Composition and Structure Data Topics or Conditions Represented
evidence instances: critically ill patients requiring acute or critical care in ICU/PICU/NICU; purposes address 'acute illness', 'critical care illness'; tasks include 'characterize acute and critical care illness patterns', 'predict complications among patients with acute or critical illness', 'measure treatment response'; addressing_gaps: lack of diverse multi-center critical care datasets
quality Clear topic focus on acute and critical care illness, ICU patient populations, complications, treatment response; purposes and tasks provide specific research questions (illness characterization, complication prediction, treatment measurement)
semantic Topics semantically coherent around critical care medicine, acute illness, ICU outcomes; multi-modal data (EHR, waveforms, imaging, notes, EEG) appropriate for comprehensive critical illness characterization
1/5 R20 Q15 (Technical Documentation) Human Subject Representation
level General human data with limited demographic details
evidence instances: critically ill patients in ICU/PICU/NICU settings, 14 hospitals, 45,000 unique admissions current, 50,000 released, 100,000+ target. subpopulations: by hospital (14 hospitals) and ICU type (ICU, PICU, NICU). No detailed demographics (age ranges, sex, race/ethnicity distributions).
quality Describes patient population and setting (critically ill, ICU/PICU/NICU) with quantitative details (50K admissions, 14 sites) but lacks detailed demographic breakdowns. Mentions diversity and Social Determinants of Health in purposes but not in instances/subpopulations.
correctness population_counts: 45K unique, 50K released, 100K+ target are plausible and internally consistent; site_count: 14 hospitals consistent across fields; population_type: critically ill ICU/PICU/NICU appropriate for critical care dataset
consistency instance_counts_consistent: 45K unique admissions and 50K total admissions (some patients may have multiple admissions); subpopulation_coverage: ICU/PICU/NICU subpopulations match instance description; diversity_claims_not_quantified: purposes mention diversity but no demographic breakdowns provided
0/1 R20 Q5 (Structural Completeness) Data File Size Availability
level Fail - no bytes field
evidence instances: 1 instance entry describing patient population. No bytes field present in distributions. Size mentioned narratively in descriptions (23TB waveform, 1.6 billion rows OMOP, 50,000 admissions)
quality D4D-core schema lacks bytes field. Quantitative size information embedded in description text but not structured metadata. Core schema limitation.
correctness instance_count_accuracy: 50,000 admissions mentioned, 100,000+ target; narrative_sizes: 23TB waveform, 1.6B rows OMOP mentioned in descriptions
consistency size_claims_across_fields: consistent 50K current, 100K+ target across multiple fields; note: D4D-core schema does not include bytes or structured size fields
subpopulations

    subpopulations:
    - id: chorus:subpop:1
      name: Critically ill patients by hospital
      description: 'Patients distributed across 14 different hospitals within the CHoRUS network spanning
        20 academic centers in the United States, ensuring geographic and institutional diversity in critical
        care settings.

        '
    - id: chorus:subpop:2
      name: ICU, PICU, and NICU patients
      description: 'Patients admitted to intensive care units (ICU), pediatric intensive care units (PICU),
        and neonatal intensive care units (NICU) across contributing hospitals, with 50,000 patient admissions
        in the current released dataset.

        '

⚠ medium R20 · completeness
issue Demographic diversity claims not quantified in instances/subpopulations
fields instances, subpopulations, purposes
fix Purposes mention diversity and Social Determinants of Health but no demographic breakdowns (age, sex, race/ethnicity) in instances. Consider adding demographic subpopulations.
✓ 1/1 R10 5.Data Composition and Structure Cohort or Subpopulations Characteristics Described
evidence subpopulations with 2 entries: (1) Patients distributed across 14 hospitals within CHoRUS network spanning 20 academic centers in US ensuring geographic/institutional diversity; (2) ICU, PICU, NICU patients with 50,000 admissions in current release; instances: 14 hospitals, 45K+ unique admissions (Aug 2025), 50K patient admissions (ICU/PICU/NICU), target 100K+ critically ill patients
quality Clear subpopulation descriptions with geographic diversity (14 hospitals, 20 academic centers), care setting diversity (ICU/PICU/NICU), and scale (50K current, 100K+ target); instances provide additional quantitative details
semantic Subpopulation characteristics semantically rich with specific institutional diversity, patient type diversity, and enrollment targets
1/5 R20 Q15 (Technical Documentation) Human Subject Representation
level General human data with limited demographic details
evidence instances: critically ill patients in ICU/PICU/NICU settings, 14 hospitals, 45,000 unique admissions current, 50,000 released, 100,000+ target. subpopulations: by hospital (14 hospitals) and ICU type (ICU, PICU, NICU). No detailed demographics (age ranges, sex, race/ethnicity distributions).
quality Describes patient population and setting (critically ill, ICU/PICU/NICU) with quantitative details (50K admissions, 14 sites) but lacks detailed demographic breakdowns. Mentions diversity and Social Determinants of Health in purposes but not in instances/subpopulations.
correctness population_counts: 45K unique, 50K released, 100K+ target are plausible and internally consistent; site_count: 14 hospitals consistent across fields; population_type: critically ill ICU/PICU/NICU appropriate for critical care dataset
consistency instance_counts_consistent: 45K unique admissions and 50K total admissions (some patients may have multiple admissions); subpopulation_coverage: ICU/PICU/NICU subpopulations match instance description; diversity_claims_not_quantified: purposes mention diversity but no demographic breakdowns provided
sensitive_elements

    sensitive_elements:
    - id: chorus:sensitive:1
      name: Protected health information - clinical EHR data
      description: 'Complete electronic health records including demographics, diagnoses, procedures, medications,
        nursing documentation, and clinical notes for critically ill patients. Contains protected health information
        subject to HIPAA and institutional privacy requirements. De-identified before release through OMOP
        transformation and privacy scanning tools.

        '
    - id: chorus:sensitive:2
      name: Physiological monitoring and EEG waveform data
      description: 'Continuous waveform telemetry from bedside monitors and EEG recordings capturing detailed
        physiological states of critically ill patients. May reveal sensitive health conditions and treatment
        responses. Stored in WFDB format (telemetry) and EDF+/Persyst formats (EEG) with controlled access.

        '
    - id: chorus:sensitive:3
      name: Medical imaging data
      description: 'Diagnostic imaging studies in DICOM format from hospital PACS systems. De-identification
        in process to remove embedded patient information while preserving clinical utility. As of August
        2025, 1,000 images available with de-id in process for larger cohort (7,642 admissions with radiology
        data total).

        '
    - id: chorus:sensitive:4
      name: Social Determinants of Health data
      description: 'Contextual factors including geographic information (distance to hospital) and social
        determinants of health data. Collected to support health equity research while maintaining patient
        privacy through privacy-preserving transformations.

        '

✓ 1/1 R10 9.Dataset Evaluation and Limitations Disclosure Sensitive Content and Warnings Provided
evidence sensitive_elements with 4 entries: (1) Protected health information - clinical EHR data (complete EHRs including demographics, diagnoses, procedures, medications, nursing documentation, clinical notes; subject to HIPAA and institutional privacy requirements; de-identified before release via OMOP transformation and privacy scanning); (2) Physiological monitoring and EEG waveform data (continuous waveforms revealing sensitive health conditions and treatment responses; WFDB/EDF+/Persyst formats with controlled access); (3) Medical imaging data (diagnostic imaging in DICOM with embedded patient information; de-id in process); (4) Social Determinants of Health data (geographic information, SDOH data; privacy-preserving transformations applied)
quality Comprehensive sensitive_elements documentation identifying all sensitive data types (PHI in EHR, physiological states in waveforms, diagnostic information in imaging, geographic/SDOH data) with privacy protections described for each; no content_warnings field in D4D-core but sensitive_elements serves similar purpose
semantic Sensitive content identification semantically appropriate for clinical dataset: PHI, physiological data, diagnostic imaging, and SDOH all require privacy protections; descriptions link sensitivity to applicable regulations (HIPAA) and mitigation strategies (de-identification, controlled access)
known_biases

    known_biases:
    - id: chorus:bias:1
      name: Academic medical center population bias
      description: 'Data collected exclusively from academic medical centers (14 acquisition centers across
        20 academic institutions), which may not represent community hospital or rural care settings. Patient
        populations at academic centers may differ systematically from the broader critically ill patient
        population in terms of disease severity, demographics, and treatment patterns.

        '
    - id: chorus:bias:2
      name: Retrospective data collection bias
      description: 'Retrospective collection from hospital electronic health records introduces potential
        selection biases based on documentation practices, coding patterns, and clinical workflows that vary
        across the 14 acquisition centers. Documentation completeness and accuracy may vary by site, modality,
        and time period.

        '

✓ 1/1 R10 5.Data Composition and Structure Data Quality Issues and Anomalies Documented
evidence known_limitations with 2 entries: (1) Dataset actively growing, not yet complete (45K current vs 100K+ target, EEG extraction in process, imaging de-id in process, notes stored locally); (2) Controlled access restricts open research (DUA required, enclave imposes computational constraints); known_biases with 2 entries: (1) Academic medical center population bias (may not represent community/rural settings); (2) Retrospective data collection bias (documentation practices vary by site/modality/time); missing_data_documentation: Incomplete modality coverage across sites (EEG/imaging in process, notes local-only)
quality Comprehensive quality documentation: known limitations address dataset completion status and access constraints; known biases address selection bias (academic centers) and documentation bias (retrospective collection); missing data documentation addresses modality coverage gaps
semantic Quality issues and biases semantically appropriate for multi-institutional retrospective clinical data collection; transparency about ongoing expansion and site-to-site variability demonstrates data quality awareness
✓ 1/1 R10 9.Dataset Evaluation and Limitations Disclosure Systematic Biases Identified and Described
evidence known_biases with 2 entries: (1) Academic medical center population bias (data from 14 acquisition centers across 20 academic institutions may not represent community hospital or rural care settings; patient populations at academic centers may differ in disease severity, demographics, treatment patterns); (2) Retrospective data collection bias (introduces selection biases based on documentation practices, coding patterns, clinical workflows varying across 14 centers; documentation completeness/accuracy may vary by site, modality, time period)
quality Explicit known_biases section identifying systematic biases with clear descriptions of mechanisms (academic vs. community/rural settings, retrospective documentation practices) and potential impacts (disease severity differences, demographics, treatment patterns, documentation completeness); future_use_impacts also mentions 'risk of algorithmic bias propagation'
semantic Biases semantically plausible for multi-institutional academic medical center dataset: academic center selection bias well-documented in health services research, retrospective documentation bias standard concern for EHR-based research
known_limitations

    known_limitations:
    - id: chorus:limitation:1
      name: Dataset still actively growing and not yet complete
      description: 'As of August 2025, the dataset covers 14 hospitals with over 45,000 unique admissions
        and 50,000 released (ICU, PICU, NICU); target exceeds 100,000 critically ill patients. EEG extraction
        and full imaging de-identification are still in process. Clinical notes are stored locally at sites
        with only tokens available in the enclave. Cohort coverage varies by data modality and ongoing collection
        continues through November 2026.

        '
    - id: chorus:limitation:2
      name: Controlled access restricts open research
      description: 'All data requires signed data use agreement and institutional email registration, limiting
        accessibility for researchers at institutions without established data sharing agreements. Access
        through secure enclave (Azure-based infrastructure) imposes computational and logistical constraints.

        '

✓ 1/1 R10 5.Data Composition and Structure Data Quality Issues and Anomalies Documented
evidence known_limitations with 2 entries: (1) Dataset actively growing, not yet complete (45K current vs 100K+ target, EEG extraction in process, imaging de-id in process, notes stored locally); (2) Controlled access restricts open research (DUA required, enclave imposes computational constraints); known_biases with 2 entries: (1) Academic medical center population bias (may not represent community/rural settings); (2) Retrospective data collection bias (documentation practices vary by site/modality/time); missing_data_documentation: Incomplete modality coverage across sites (EEG/imaging in process, notes local-only)
quality Comprehensive quality documentation: known limitations address dataset completion status and access constraints; known biases address selection bias (academic centers) and documentation bias (retrospective collection); missing data documentation addresses modality coverage gaps
semantic Quality issues and biases semantically appropriate for multi-institutional retrospective clinical data collection; transparency about ongoing expansion and site-to-site variability demonstrates data quality awareness
✓ 1/1 R10 9.Dataset Evaluation and Limitations Disclosure Known Limitations Documented
evidence known_limitations with 2 entries: (1) Dataset actively growing, not yet complete (Aug 2025: 14 hospitals, 45K+ unique admissions, 50K released vs target 100K+; EEG extraction in process, imaging de-id in process, notes stored locally with only tokens in enclave; cohort coverage varies by modality; ongoing collection through Nov 2026); (2) Controlled access restricts open research (DUA required, institutional email registration, secure enclave imposes computational/logistical constraints)
quality Explicit known_limitations section with clear documentation of dataset completeness status, ongoing expansion timeline, modality-specific processing status, and access constraints; provides actionable information for dataset users about current vs. planned coverage
semantic Limitations semantically appropriate for large-scale ongoing data generation project: transparency about growth trajectory, modality processing status, and access model constraints
✓ 1/1 R10 9.Dataset Evaluation and Limitations Disclosure Data Anomalies and Quality Issues Noted
evidence missing_data_documentation: Incomplete modality coverage across sites (not all modalities available from all 14 centers; EEG extraction and imaging de-id in process as of Aug 2025; clinical notes stored locally with only tokens in enclave; cohort coverage varies by modality during ongoing collection); cleaning_strategies mention cross-site data quality checks via CHoRUSReports; known_limitations mention documentation completeness/accuracy varying by site/modality/time
quality Data quality issues documented: incomplete modality coverage across sites, ongoing processing for EEG/imaging, federated note storage (only tokens centralized), cross-site variability in documentation practices; cleaning_strategies describe quality assurance via CHoRUSReports characterization
semantic Quality issues semantically appropriate for large-scale multi-institutional data aggregation: site-to-site variability in data availability and completeness expected, ongoing processing status provides transparency about dataset maturity
confidential_elements

    confidential_elements:
    - id: chorus:confidential:1
      name: De-identified clinical data under controlled access
      description: 'All data distributed under controlled access model requiring institutional email registration
        and signed licensing agreement. Data de-identified before release. Full clinical notes stored locally
        at sites; only OHNLP tokens available in enclave. Data use agreement prohibits re-identification attempts.

        '

✓ 1/1 R10 2.Dataset Access and Retrieval Regulatory Restrictions and Confidentiality Level Specified
evidence regulatory_restrictions: HIPAA compliance, 45 CFR 46 (Common Rule) for human subjects research, institutional data privacy regulations at 14 sites, NIH Bridge2AI ethical AI requirements, export control regulations for international users; confidential_elements: De-identified clinical data under controlled access model
quality Comprehensive regulatory framework with HIPAA, Common Rule, institutional requirements, and ethical AI standards; confidentiality level clearly specified as de-identified with controlled access
semantic Regulatory restrictions appropriate for multi-institutional clinical research dataset; export control mention suggests awareness of international data sharing constraints
acquisition_methods

    acquisition_methods:
    - id: chorus:acquisition:1
      name: Structured EHR data via OMOP transformation
      description: 'Demographics, medication administration (dosing time-stamped upon each infusion change
        or dose administration), procedures, nursing flowsheets (high-frequency documentation), and diagnoses
        acquired from electronic health records and standardized to OMOP Common Data Model. Controlled access
        with published OMOP schema metadata. Contains 1.6 billion rows of OMOP data as of August 2025.

        '
    - id: chorus:acquisition:2
      name: Clinical notes via OHNLP tokenization
      description: 'Clinical notes extracted from EHR systems and tokenized using OHNLP (Open Health Natural
        Language Processing) toolkit. Stored locally at sites except for tokens. Controlled access planned
        with OHNLP open source schema metadata.

        '
    - id: chorus:acquisition:3
      name: Medical imaging via DICOM from PACS
      description: 'Imaging data acquired from hospital PACS systems in DICOM format. De-identification in
        process. Planned controlled access with published DICOM schema metadata.

        '
    - id: chorus:acquisition:4
      name: Waveform telemetry via WFDB from bedside monitors
      description: 'Bedside monitor waveform data acquired through gateway and middleware systems. Stored
        in WFDB format with controlled access and published PhysioNet schema (extended) metadata. 23 TB available
        as of August 2025.

        '
    - id: chorus:acquisition:5
      name: EEG waveforms from hospital databases
      description: 'EEG recordings from hospital databases in EDF+ and Persyst formats. Extraction in process.
        Planned controlled access with open source EDF+ and Persyst schema metadata.

        '

✓ 1/1 R10 8.Technical Transparency (Data Collection and Processing) Data Acquisition Methods Listed
evidence acquisition_methods with 5 entries: (1) Structured EHR via OMOP transformation (demographics, medication administration with dosing time-stamps, procedures, nursing flowsheets with high-frequency documentation, diagnoses, 1.6B rows); (2) Clinical notes via OHNLP tokenization (stored locally at sites except tokens, controlled access planned); (3) Medical imaging via DICOM from PACS (de-identification in process, planned controlled access); (4) Waveform telemetry via WFDB from bedside monitors (gateway/middleware systems, 23TB, controlled access); (5) EEG waveforms from hospital databases (EDF+/Persyst formats, extraction in process)
quality Comprehensive acquisition methods covering all modalities with technical specifics (OMOP transformation, OHNLP tokenization, DICOM from PACS, WFDB from bedside monitors, EDF+/Persyst from databases); includes data volumes (1.6B rows, 23TB) and processing status (de-id in process, extraction in process)
semantic Acquisition methods semantically detailed with standard clinical informatics workflows: OMOP transformation for EHR harmonization, OHNLP for clinical text NLP, DICOM for medical imaging, WFDB for physiological signals, EDF+ for neurophysiology
4/5 R20 Q12 (Technical Documentation) Collection Protocol Clarity
level Comprehensive collection protocol
evidence acquisition_methods: 5 methods (EHR via OMOP, notes via OHNLP, imaging via DICOM/PACS, waveform via WFDB, EEG). collection_mechanisms: 4 mechanisms (retrospective EHR extraction, waveform telemetry capture, medical imaging from PACS, EEG extraction). data_collectors: 14 data acquisition centers with site status tracking via GitHub and Google Forms. collection_timeframes: Sept 1, 2022 to Nov 30, 2026.
quality Very good collection protocol documentation with detailed acquisition methods, mechanisms, collector details, and timeframes. Specifies 14 acquisition centers and provides specific technical details for each data modality.
correctness date_validity: Sept 2022 to Nov 2026 is valid timeline; site_count: 14 acquisition centers across 20 academic institutions is internally consistent; acquisition_methods_appropriateness: retrospective EHR extraction appropriate for this dataset type
consistency timeframe_matches_funding_dates: True; site_count_consistent: 14 acquisition centers mentioned consistently across fields; acquisition_methods_match_distributions: 5 acquisition methods correspond to 5 distribution formats
collection_mechanisms

    collection_mechanisms:
    - id: chorus:collection:1
      name: Retrospective EHR extraction
      description: 'Retrospective data collection from electronic health record systems at 14 data acquisition
        centers. Data extracted includes demographics, medication administration (dosing time-stamped upon
        each infusion change or dose administration), procedures, nursing flowsheets (high-frequency documentation),
        diagnoses, and clinical notes. Standardized to OMOP Common Data Model.

        '
    - id: chorus:collection:2
      name: Waveform telemetry capture
      description: 'Continuous waveform telemetry data captured from bedside monitors through gateway and
        middleware systems at each contributing hospital. Stored in WFDB (WaveForm DataBase) format following
        extended PhysioNet schema. Contains 23 TB of waveform data as of August 2025.

        '
    - id: chorus:collection:3
      name: Medical imaging acquisition from PACS
      description: 'Medical imaging data acquired from hospital Picture Archiving and Communication Systems
        (PACS) and stored in DICOM format with comprehensive metadata following DICOM schema. 7,642 admissions
        have radiology data; 1,000 images currently available.

        '
    - id: chorus:collection:4
      name: EEG recording extraction
      description: 'Electroencephalography recordings extracted from hospital EEG databases in EDF+ (European
        Data Format) and Persyst formats with metadata following open source schemas. Extraction in process
        as of August 2025.

        '

✓ 1/1 R10 8.Technical Transparency (Data Collection and Processing) Collection Mechanisms and Settings Described
evidence collection_mechanisms with 4 entries: (1) Retrospective EHR extraction at 14 centers (demographics, medications with dosing time-stamps, procedures, nursing flowsheets with high-frequency documentation, diagnoses, clinical notes, standardized to OMOP CDM); (2) Waveform telemetry capture from bedside monitors via gateway/middleware systems (23TB in WFDB format); (3) Medical imaging acquisition from PACS (7,642 admissions, 1K images available, DICOM format); (4) EEG recording extraction from hospital databases (EDF+/Persyst formats, extraction in process); collection_timeframes: Sept 1, 2022 - Nov 30, 2026
quality Detailed collection mechanisms for each modality with specific technical details (time-stamped medications, high-frequency nursing flowsheets, gateway/middleware systems for waveforms, PACS for imaging); temporal context provided (Sept 2022 - Nov 2026); data_collectors identify 14 acquisition centers with GitHub tracking and Chorus_SOP protocols
semantic Collection mechanisms semantically appropriate for multi-modal clinical data: retrospective EHR extraction standard for research datasets, bedside monitor gateways standard for waveform capture, PACS standard for imaging retrieval, hospital databases standard for EEG archives
4/5 R20 Q12 (Technical Documentation) Collection Protocol Clarity
level Comprehensive collection protocol
evidence acquisition_methods: 5 methods (EHR via OMOP, notes via OHNLP, imaging via DICOM/PACS, waveform via WFDB, EEG). collection_mechanisms: 4 mechanisms (retrospective EHR extraction, waveform telemetry capture, medical imaging from PACS, EEG extraction). data_collectors: 14 data acquisition centers with site status tracking via GitHub and Google Forms. collection_timeframes: Sept 1, 2022 to Nov 30, 2026.
quality Very good collection protocol documentation with detailed acquisition methods, mechanisms, collector details, and timeframes. Specifies 14 acquisition centers and provides specific technical details for each data modality.
correctness date_validity: Sept 2022 to Nov 2026 is valid timeline; site_count: 14 acquisition centers across 20 academic institutions is internally consistent; acquisition_methods_appropriateness: retrospective EHR extraction appropriate for this dataset type
consistency timeframe_matches_funding_dates: True; site_count_consistent: 14 acquisition centers mentioned consistently across fields; acquisition_methods_match_distributions: 5 acquisition methods correspond to 5 distribution formats
collection_timeframes

    collection_timeframes:
    - id: chorus:timeframe:1
      name: Project data collection period
      description: 'Retrospective and ongoing data collection from September 1, 2022 through November 30,
        2026 (project end date with approved no-cost extension). As of August 2025, data covers 14 different
        hospitals with over 45,000 unique admissions. Target enrollment exceeds 100,000 critically ill patients
        by project completion.

        '

✓ 1/1 R10 6.Data Provenance and Version Tracking Update Schedule or Frequency Indicated
evidence updates: Dataset updated continuously as data collection progresses at 14 acquisition centers; project timeline extends through November 30, 2026 (approved no-cost extension); regular status updates tracked through GitHub project management system; collection_timeframes: September 1, 2022 through November 30, 2026
quality Update schedule documented: continuous updates during ongoing collection (Sept 2022 - Nov 2026); regular status tracking via GitHub; temporal milestones provided (current Aug 2025 snapshot, end date Nov 2026)
semantic Update schedule semantically clear: continuous expansion during project period with regular status tracking; end date (Nov 2026) provides timeline boundary
✓ 1/1 R10 8.Technical Transparency (Data Collection and Processing) Collection Mechanisms and Settings Described
evidence collection_mechanisms with 4 entries: (1) Retrospective EHR extraction at 14 centers (demographics, medications with dosing time-stamps, procedures, nursing flowsheets with high-frequency documentation, diagnoses, clinical notes, standardized to OMOP CDM); (2) Waveform telemetry capture from bedside monitors via gateway/middleware systems (23TB in WFDB format); (3) Medical imaging acquisition from PACS (7,642 admissions, 1K images available, DICOM format); (4) EEG recording extraction from hospital databases (EDF+/Persyst formats, extraction in process); collection_timeframes: Sept 1, 2022 - Nov 30, 2026
quality Detailed collection mechanisms for each modality with specific technical details (time-stamped medications, high-frequency nursing flowsheets, gateway/middleware systems for waveforms, PACS for imaging); temporal context provided (Sept 2022 - Nov 2026); data_collectors identify 14 acquisition centers with GitHub tracking and Chorus_SOP protocols
semantic Collection mechanisms semantically appropriate for multi-modal clinical data: retrospective EHR extraction standard for research datasets, bedside monitor gateways standard for waveform capture, PACS standard for imaging retrieval, hospital databases standard for EEG archives
4/5 R20 Q12 (Technical Documentation) Collection Protocol Clarity
level Comprehensive collection protocol
evidence acquisition_methods: 5 methods (EHR via OMOP, notes via OHNLP, imaging via DICOM/PACS, waveform via WFDB, EEG). collection_mechanisms: 4 mechanisms (retrospective EHR extraction, waveform telemetry capture, medical imaging from PACS, EEG extraction). data_collectors: 14 data acquisition centers with site status tracking via GitHub and Google Forms. collection_timeframes: Sept 1, 2022 to Nov 30, 2026.
quality Very good collection protocol documentation with detailed acquisition methods, mechanisms, collector details, and timeframes. Specifies 14 acquisition centers and provides specific technical details for each data modality.
correctness date_validity: Sept 2022 to Nov 2026 is valid timeline; site_count: 14 acquisition centers across 20 academic institutions is internally consistent; acquisition_methods_appropriateness: retrospective EHR extraction appropriate for this dataset type
consistency timeframe_matches_funding_dates: True; site_count_consistent: 14 acquisition centers mentioned consistently across fields; acquisition_methods_match_distributions: 5 acquisition methods correspond to 5 distribution formats
data_collectors

    data_collectors:
    - id: chorus:collector:1
      name: CHoRUS Data Acquisition Centers
      description: '14 data acquisition centers across 20 academic institutions in the United States responsible
        for extracting and contributing clinical data (EHR, waveforms, imaging, notes, EEG) from their hospital
        systems. Site statuses tracked through GitHub interface and Google Form submissions. Data Acquisition
        sub-team manages extraction per Chorus_SOP standard operating protocols.

        '

4/5 R20 Q12 (Technical Documentation) Collection Protocol Clarity
level Comprehensive collection protocol
evidence acquisition_methods: 5 methods (EHR via OMOP, notes via OHNLP, imaging via DICOM/PACS, waveform via WFDB, EEG). collection_mechanisms: 4 mechanisms (retrospective EHR extraction, waveform telemetry capture, medical imaging from PACS, EEG extraction). data_collectors: 14 data acquisition centers with site status tracking via GitHub and Google Forms. collection_timeframes: Sept 1, 2022 to Nov 30, 2026.
quality Very good collection protocol documentation with detailed acquisition methods, mechanisms, collector details, and timeframes. Specifies 14 acquisition centers and provides specific technical details for each data modality.
correctness date_validity: Sept 2022 to Nov 2026 is valid timeline; site_count: 14 acquisition centers across 20 academic institutions is internally consistent; acquisition_methods_appropriateness: retrospective EHR extraction appropriate for this dataset type
consistency timeframe_matches_funding_dates: True; site_count_consistent: 14 acquisition centers mentioned consistently across fields; acquisition_methods_match_distributions: 5 acquisition methods correspond to 5 distribution formats
sampling_strategies

    sampling_strategies:
    - id: chorus:sampling:1
      name: Multi-center federated sampling
      description: 'Federated access enables sampling methods to ensure a balanced and diverse cohort across
        20 academic centers (14 data acquisition centers). Legal framework established for collecting data
        at scale with sampling to ensure comprehensive sets of patient conditions and clinical treatment strategies.
        Community-facing ethics focus groups conducted to determine what data is appropriate for public sharing.

        '

no field-level feedback matched
raw_data_sources

    raw_data_sources:
    - id: chorus:rawsource:1
      name: Hospital EHR systems at 14 acquisition centers
      description: 'Electronic health record systems at 14 data acquisition centers across the United States
        providing structured clinical data (demographics, medications, procedures, diagnoses, nursing flowsheets)
        in diverse institutional formats prior to OMOP standardization.

        '
      source_description: 'Institutional EHR systems (diverse formats) at 14 academic data acquisition centers
        in the United States, containing structured clinical data for critically ill patients.

        '
    - id: chorus:rawsource:2
      name: Bedside monitoring systems (waveform)
      description: 'Proprietary bedside patient monitoring systems at participating hospitals serving as the
        raw source for continuous waveform telemetry data prior to WFDB standardization.

        '
      source_description: 'Proprietary bedside patient monitoring gateway and middleware systems at 14 participating
        hospitals, capturing continuous physiological waveform telemetry for ICU patients.

        '
    - id: chorus:rawsource:3
      name: Hospital PACS systems (imaging)
      description: 'Picture Archiving and Communication Systems at participating hospitals serving as the
        raw source for medical imaging data in DICOM format prior to de-identification.

        '
      source_description: 'Hospital Picture Archiving and Communication Systems (PACS) at participating institutions,
        containing radiology imaging data (CT, X-ray, MRI, and other modalities) in DICOM format.

        '
    - id: chorus:rawsource:4
      name: Hospital EEG databases
      description: 'Hospital electroencephalography databases serving as raw source for EEG recordings in
        EDF+ and Persyst formats prior to extraction and standardization.

        '
      source_description: 'Hospital electroencephalography recording systems and databases at participating
        sites, containing EEG recordings in EDF+ (European Data Format) and Persyst formats.

        '

✗ 0/1 R10 6.Data Provenance and Version Tracking Provenance and Source Derivation Documented
evidence No was_derived_from or release_notes fields in D4D-core; raw_data_sources with 4 entries describe institutional EHR systems, bedside monitoring systems, hospital PACS, hospital EEG databases; raw_sources describe 'unprocessed clinical data from 14 acquisition centers in diverse institutional formats'
quality D4D-core schema lacks was_derived_from and release_notes fields; provenance information present in raw_data_sources and raw_sources (14 institutional EHR systems, bedside monitors, PACS, EEG databases) but not structured derivation metadata
semantic Source provenance documented in raw_data_sources but lacks structured derivation lineage; full D4D schema includes was_derived_from for formal provenance chains
missing_data_documentation

    missing_data_documentation:
    - id: chorus:missing:1
      name: Incomplete modality coverage across sites
      description: 'Not all data modalities are available from all 14 acquisition centers. EEG extraction
        and full imaging de-identification are in process as of August 2025. Clinical notes stored locally
        at sites with only OHNLP tokens available in the central enclave. Cohort coverage varies by data modality
        during ongoing data collection.

        '

✓ 1/1 R10 5.Data Composition and Structure Data Quality Issues and Anomalies Documented
evidence known_limitations with 2 entries: (1) Dataset actively growing, not yet complete (45K current vs 100K+ target, EEG extraction in process, imaging de-id in process, notes stored locally); (2) Controlled access restricts open research (DUA required, enclave imposes computational constraints); known_biases with 2 entries: (1) Academic medical center population bias (may not represent community/rural settings); (2) Retrospective data collection bias (documentation practices vary by site/modality/time); missing_data_documentation: Incomplete modality coverage across sites (EEG/imaging in process, notes local-only)
quality Comprehensive quality documentation: known limitations address dataset completion status and access constraints; known biases address selection bias (academic centers) and documentation bias (retrospective collection); missing data documentation addresses modality coverage gaps
semantic Quality issues and biases semantically appropriate for multi-institutional retrospective clinical data collection; transparency about ongoing expansion and site-to-site variability demonstrates data quality awareness
✓ 1/1 R10 9.Dataset Evaluation and Limitations Disclosure Data Anomalies and Quality Issues Noted
evidence missing_data_documentation: Incomplete modality coverage across sites (not all modalities available from all 14 centers; EEG extraction and imaging de-id in process as of Aug 2025; clinical notes stored locally with only tokens in enclave; cohort coverage varies by modality during ongoing collection); cleaning_strategies mention cross-site data quality checks via CHoRUSReports; known_limitations mention documentation completeness/accuracy varying by site/modality/time
quality Data quality issues documented: incomplete modality coverage across sites, ongoing processing for EEG/imaging, federated note storage (only tokens centralized), cross-site variability in documentation practices; cleaning_strategies describe quality assurance via CHoRUSReports characterization
semantic Quality issues semantically appropriate for large-scale multi-institutional data aggregation: site-to-site variability in data availability and completeness expected, ongoing processing status provides transparency about dataset maturity
raw_sources

    raw_sources:
    - id: chorus:raw:1
      name: Raw multi-institutional clinical data
      description: 'Unprocessed clinical data from 14 acquisition centers in diverse institutional formats
        including proprietary EHR exports, raw waveform streams from bedside monitors, DICOM images from PACS,
        and clinical notes in free-text format prior to any standardization or de-identification processing.

        '

✗ 0/1 R10 6.Data Provenance and Version Tracking Provenance and Source Derivation Documented
evidence No was_derived_from or release_notes fields in D4D-core; raw_data_sources with 4 entries describe institutional EHR systems, bedside monitoring systems, hospital PACS, hospital EEG databases; raw_sources describe 'unprocessed clinical data from 14 acquisition centers in diverse institutional formats'
quality D4D-core schema lacks was_derived_from and release_notes fields; provenance information present in raw_data_sources and raw_sources (14 institutional EHR systems, bedside monitors, PACS, EEG databases) but not structured derivation metadata
semantic Source provenance documented in raw_data_sources but lacks structured derivation lineage; full D4D schema includes was_derived_from for formal provenance chains
preprocessing_strategies

    preprocessing_strategies:
    - id: chorus:preproc:1
      name: OMOP Common Data Model transformation
      description: 'All structured electronic health record data standardized to the OMOP (Observational Medical
        Outcomes Partnership) Common Data Model. Ensures interoperability and enables use of OHDSI (Observational
        Health Data Sciences and Informatics) tool stack for analysis. Data from all 14 acquisition centers
        unified through this transformation. Validated semantic mappings for connecting clinical data maintained
        in chorus-mapping repository. Characterization reports returned to contributing sites via CHoRUSReports.

        '
    - id: chorus:preproc:2
      name: Clinical note tokenization via OHNLP
      description: 'Clinical notes processed using OHNLP (Open Health Natural Language Processing) toolkit
        for extraction and tokenization. Protects patient privacy while enabling natural language processing
        and analysis of clinical text data. Full notes stored locally at sites; only tokens available in enclave.

        '
    - id: chorus:preproc:3
      name: Waveform standardization to WFDB format
      description: 'Waveform telemetry data from diverse bedside monitoring systems standardized to WFDB (WaveForm
        DataBase) format following extended PhysioNet schema. Scripts available in chorus_waveform repository.

        '
    - id: chorus:preproc:4
      name: Medical imaging de-identification
      description: 'DICOM imaging data undergoing de-identification process to remove patient identifiable
        information from image metadata while preserving clinical utility and image quality. Privacy scan
        tool (privacy_scan_tool) used for medical records privacy scanning.

        '
    - id: chorus:preproc:5
      name: Re-identification limitation transformations
      description: 'Data transformed using approaches that limit re-identification while maintaining analytical
        utility. Multiple preprocessing strategies employed to protect patient privacy across all data modalities
        per HIPAA and institutional requirements. Geocoding via DeGauss (UF-Geocoding tool) for OMOP Location
        entities.

        '

⚠ low R20 · consistency
issue Software tools mentioned by name but lacking version numbers and URLs
fields preprocessing_strategies, cleaning_strategies
fix Add version numbers for OHNLP toolkit, OHDSI tools, privacy_scan_tool, DeGauss. Include GitHub URLs for chorus-mapping and chorus_waveform repositories.
✓ 1/1 R10 10.Cross-Platform and Community Integration Community Standards or Schema Conformance
evidence OMOP Common Data Model (OHDSI standard for observational health research), DICOM (medical imaging standard), WFDB (PhysioNet waveform standard), OHNLP (open source clinical NLP), EDF+ (European Data Format for EEG), Persyst (EEG format); external_resources link to OHDSI community (https://www.ohdsi.org/); preprocessing_strategies mention 'validated semantic mappings' in chorus-mapping repository with 'clinical validation SOP'
quality Comprehensive community standard conformance across all modalities: OMOP CDM (OHDSI ecosystem), DICOM (medical imaging interoperability), WFDB (PhysioNet physiological signal exchange), OHNLP (clinical NLP), EDF+ (neurophysiology); validated semantic mappings maintained in GitHub repository; external link to OHDSI community for tool ecosystem
semantic Standards conformance semantically robust: OMOP CDM enables cross-study observational research via OHDSI tools, DICOM enables imaging workflow integration, WFDB enables PhysioNet tool compatibility, OHNLP enables NLP research; validated mappings demonstrate quality assurance for standard conformance
✗ 0/1 R10 3.Data Reuse and Interoperability Schema or Ontology Conformance Stated
evidence No conforms_to or conforms_to_schema fields in D4D-core; preprocessing_strategies and distribution_formats describe OMOP, DICOM, WFDB, OHNLP, EDF+/Persyst schemas but not in dedicated conformance field
quality D4D-core schema lacks conforms_to field; schema conformance information present in descriptive text (OMOP CDM, DICOM schema, PhysioNet WFDB schema, OHNLP schema, EDF+ schema) but not structured metadata
semantic Full D4D schema includes conforms_to field; D4D-core subset does not expose this structured metadata
✓ 1/1 R10 4.Ethical Use and Privacy Safeguards Privacy Protections Beyond Deidentification
evidence Controlled access model requiring DUA and institutional email despite de-identification; privacy_scan_tool for medical records privacy scanning; OHNLP tokenization with full notes stored locally at sites (only tokens in enclave); preprocessing_strategies mention 're-identification limitation transformations'; legal framework and community ethics focus groups for privacy governance
quality Multi-layered privacy protections: de-identification + controlled access + DUA + privacy scanning tools + federated data storage (notes remain local) + re-identification limitation transformations; community ethics focus groups determine appropriate data sharing
semantic Privacy protections appropriate for sensitive clinical data; defense-in-depth approach (de-id + access controls + legal agreements) reflects best practices for health data sharing
✓ 1/1 R10 8.Technical Transparency (Data Collection and Processing) Preprocessing, Cleaning, and Labeling Strategies
evidence preprocessing_strategies with 5 entries: OMOP CDM transformation with validated semantic mappings (chorus-mapping repo), OHNLP tokenization for privacy, WFDB standardization (chorus_waveform repo), DICOM de-identification via privacy_scan_tool, re-identification limitation transformations + DeGauss geocoding; cleaning_strategies with 2 entries: multi-center harmonization with validated semantic mappings and SOPs (CHoRUSReports for quality checks), schema compliance validation across all modalities; labeling_strategies: visualization/annotation environment for prediction targets with quality control via Chorus_SOP
quality Detailed preprocessing across all modalities with specific tools (OMOP transformation, OHNLP, WFDB, privacy_scan_tool, DeGauss); cleaning strategies address cross-site harmonization and schema compliance; labeling strategy describes annotation workflow for prediction targets; references to GitHub repos (chorus-mapping, chorus_waveform) and SOPs (Chorus_SOP, CHoRUSReports)
semantic Preprocessing/cleaning/labeling strategies semantically appropriate for multi-institutional clinical dataset: OMOP transformation enables cross-site harmonization, validated semantic mappings ensure data quality, schema compliance validation ensures standard conformance, annotation environment supports supervised learning tasks
1/5 R20 Q10 (Metadata Quality & Content) Interoperability and Standardization
level Standard formats mentioned but no schema conformance fields
evidence distribution_formats: OMOP Common Data Model, DICOM, WFDB, OHNLP, EDF+, Persyst (all industry standards). preprocessing_strategies mention validated semantic mappings and schema compliance validation. No conforms_to or conforms_to_schema fields in core schema.
quality Uses multiple recognized industry standards (OMOP, DICOM, WFDB, OHNLP, EDF+) but core schema lacks explicit conforms_to/conforms_to_schema fields for formal schema references. Standards mentioned narratively.
correctness format_standards: OMOP, DICOM, WFDB, OHNLP, EDF+ are all recognized healthcare/clinical data standards; omop_description: correctly describes OMOP Common Data Model and OHDSI tool stack; dicom_description: correctly identifies DICOM as imaging standard; wfdb_description: correctly identifies WFDB as PhysioNet waveform format
consistency formats_match_data_types: OMOP for EHR, DICOM for imaging, WFDB for waveforms, OHNLP for notes (all appropriate); schema_validation_mentioned: preprocessing_strategies describe schema compliance validation; note: Core schema may not include conforms_to/conforms_to_schema fields
3/5 R20 Q11 (Technical Documentation) Tool and Software Transparency
level Strategies listed but limited tool details
evidence preprocessing_strategies: 5 strategies (OMOP transformation, OHNLP tokenization, WFDB standardization, DICOM de-id, re-id limitation). cleaning_strategies: 2 strategies (multi-center harmonization, schema compliance). labeling_strategies: 1 strategy (visualization and annotation environment). Software tools mentioned: privacy_scan_tool, DeGauss (UF-Geocoding), OHDSI tool stack, OHNLP toolkit, but no version numbers or URLs.
quality Good coverage of strategies across preprocessing, cleaning, and labeling. Software tools mentioned by name but lacking version numbers, GitHub links, or detailed documentation references. Core schema may not have dedicated software_and_tools field.
correctness tool_names: OHNLP, OHDSI, DeGauss, privacy_scan_tool are plausible tool names; preprocessing_methods: OMOP transformation, tokenization, WFDB conversion appropriate for described data types
consistency preprocessing_aligns_with_formats: OMOP preprocessing matches OMOP distribution format; tools_match_strategies: OHNLP toolkit matches clinical note tokenization strategy; github_references: mentions chorus-mapping and chorus_waveform repos but not full URLs
cleaning_strategies

    cleaning_strategies:
    - id: chorus:cleaning:1
      name: Multi-center data harmonization with validated semantic mappings
      description: 'Data from 14 acquisition centers harmonized through validated semantic mappings and standard
        operating protocols (SOPs). Ensures consistency and interoperability across diverse institutional
        EHR systems and clinical practices. Mappings maintained in chorus-mapping repository with clinical
        validation SOP. Cross-site data quality checks through CHoRUSReports characterization reports.

        '
    - id: chorus:cleaning:2
      name: Schema compliance validation across all modalities
      description: 'All data modalities validated against published metadata schemas (OMOP, DICOM, WFDB, OHNLP,
        EDF+, Persyst) to ensure compliance and data quality across the multi-modal dataset. Schema extensions
        documented for OMOP high-frequency nursing flowsheets.

        '

⚠ low R20 · consistency
issue Software tools mentioned by name but lacking version numbers and URLs
fields preprocessing_strategies, cleaning_strategies
fix Add version numbers for OHNLP toolkit, OHDSI tools, privacy_scan_tool, DeGauss. Include GitHub URLs for chorus-mapping and chorus_waveform repositories.
✓ 1/1 R10 8.Technical Transparency (Data Collection and Processing) Preprocessing, Cleaning, and Labeling Strategies
evidence preprocessing_strategies with 5 entries: OMOP CDM transformation with validated semantic mappings (chorus-mapping repo), OHNLP tokenization for privacy, WFDB standardization (chorus_waveform repo), DICOM de-identification via privacy_scan_tool, re-identification limitation transformations + DeGauss geocoding; cleaning_strategies with 2 entries: multi-center harmonization with validated semantic mappings and SOPs (CHoRUSReports for quality checks), schema compliance validation across all modalities; labeling_strategies: visualization/annotation environment for prediction targets with quality control via Chorus_SOP
quality Detailed preprocessing across all modalities with specific tools (OMOP transformation, OHNLP, WFDB, privacy_scan_tool, DeGauss); cleaning strategies address cross-site harmonization and schema compliance; labeling strategy describes annotation workflow for prediction targets; references to GitHub repos (chorus-mapping, chorus_waveform) and SOPs (Chorus_SOP, CHoRUSReports)
semantic Preprocessing/cleaning/labeling strategies semantically appropriate for multi-institutional clinical dataset: OMOP transformation enables cross-site harmonization, validated semantic mappings ensure data quality, schema compliance validation ensures standard conformance, annotation environment supports supervised learning tasks
✓ 1/1 R10 9.Dataset Evaluation and Limitations Disclosure Data Anomalies and Quality Issues Noted
evidence missing_data_documentation: Incomplete modality coverage across sites (not all modalities available from all 14 centers; EEG extraction and imaging de-id in process as of Aug 2025; clinical notes stored locally with only tokens in enclave; cohort coverage varies by modality during ongoing collection); cleaning_strategies mention cross-site data quality checks via CHoRUSReports; known_limitations mention documentation completeness/accuracy varying by site/modality/time
quality Data quality issues documented: incomplete modality coverage across sites, ongoing processing for EEG/imaging, federated note storage (only tokens centralized), cross-site variability in documentation practices; cleaning_strategies describe quality assurance via CHoRUSReports characterization
semantic Quality issues semantically appropriate for large-scale multi-institutional data aggregation: site-to-site variability in data availability and completeness expected, ongoing processing status provides transparency about dataset maturity
3/5 R20 Q11 (Technical Documentation) Tool and Software Transparency
level Strategies listed but limited tool details
evidence preprocessing_strategies: 5 strategies (OMOP transformation, OHNLP tokenization, WFDB standardization, DICOM de-id, re-id limitation). cleaning_strategies: 2 strategies (multi-center harmonization, schema compliance). labeling_strategies: 1 strategy (visualization and annotation environment). Software tools mentioned: privacy_scan_tool, DeGauss (UF-Geocoding), OHDSI tool stack, OHNLP toolkit, but no version numbers or URLs.
quality Good coverage of strategies across preprocessing, cleaning, and labeling. Software tools mentioned by name but lacking version numbers, GitHub links, or detailed documentation references. Core schema may not have dedicated software_and_tools field.
correctness tool_names: OHNLP, OHDSI, DeGauss, privacy_scan_tool are plausible tool names; preprocessing_methods: OMOP transformation, tokenization, WFDB conversion appropriate for described data types
consistency preprocessing_aligns_with_formats: OMOP preprocessing matches OMOP distribution format; tools_match_strategies: OHNLP toolkit matches clinical note tokenization strategy; github_references: mentions chorus-mapping and chorus_waveform repos but not full URLs
labeling_strategies

    labeling_strategies:
    - id: chorus:labeling:1
      name: Visualization and annotation environment for prediction targets
      description: 'Custom visualization and annotation environment developed to label data with targets important
        for prediction tasks in critical care AI applications. Supports labeling for characterizing acute
        illness, predicting complications, and measuring treatment response. Annotation interface for clinical
        expert labeling with quality control through review processes per Chorus_SOP standard operating procedures.

        '

✓ 1/1 R10 8.Technical Transparency (Data Collection and Processing) Preprocessing, Cleaning, and Labeling Strategies
evidence preprocessing_strategies with 5 entries: OMOP CDM transformation with validated semantic mappings (chorus-mapping repo), OHNLP tokenization for privacy, WFDB standardization (chorus_waveform repo), DICOM de-identification via privacy_scan_tool, re-identification limitation transformations + DeGauss geocoding; cleaning_strategies with 2 entries: multi-center harmonization with validated semantic mappings and SOPs (CHoRUSReports for quality checks), schema compliance validation across all modalities; labeling_strategies: visualization/annotation environment for prediction targets with quality control via Chorus_SOP
quality Detailed preprocessing across all modalities with specific tools (OMOP transformation, OHNLP, WFDB, privacy_scan_tool, DeGauss); cleaning strategies address cross-site harmonization and schema compliance; labeling strategy describes annotation workflow for prediction targets; references to GitHub repos (chorus-mapping, chorus_waveform) and SOPs (Chorus_SOP, CHoRUSReports)
semantic Preprocessing/cleaning/labeling strategies semantically appropriate for multi-institutional clinical dataset: OMOP transformation enables cross-site harmonization, validated semantic mappings ensure data quality, schema compliance validation ensures standard conformance, annotation environment supports supervised learning tasks
3/5 R20 Q11 (Technical Documentation) Tool and Software Transparency
level Strategies listed but limited tool details
evidence preprocessing_strategies: 5 strategies (OMOP transformation, OHNLP tokenization, WFDB standardization, DICOM de-id, re-id limitation). cleaning_strategies: 2 strategies (multi-center harmonization, schema compliance). labeling_strategies: 1 strategy (visualization and annotation environment). Software tools mentioned: privacy_scan_tool, DeGauss (UF-Geocoding), OHDSI tool stack, OHNLP toolkit, but no version numbers or URLs.
quality Good coverage of strategies across preprocessing, cleaning, and labeling. Software tools mentioned by name but lacking version numbers, GitHub links, or detailed documentation references. Core schema may not have dedicated software_and_tools field.
correctness tool_names: OHNLP, OHDSI, DeGauss, privacy_scan_tool are plausible tool names; preprocessing_methods: OMOP transformation, tokenization, WFDB conversion appropriate for described data types
consistency preprocessing_aligns_with_formats: OMOP preprocessing matches OMOP distribution format; tools_match_strategies: OHNLP toolkit matches clinical note tokenization strategy; github_references: mentions chorus-mapping and chorus_waveform repos but not full URLs
intended_uses

    intended_uses:
    - id: chorus:use:1
      name: AI/ML model development for critical care
      description: 'Primary intended use is development and training of artificial intelligence and machine
        learning models to characterize acute and critical care illness, predict complications, and measure
        treatment response in critically ill patients across diverse hospital settings.

        '
    - id: chorus:use:2
      name: External validation of AI models
      description: 'Provision of holdout test set accessible for model external validation to aid marketplace
        adoption of AI-developed models for implementation in acute and critical care settings.

        '
    - id: chorus:use:3
      name: Health equity and disparities research
      description: 'Studies examining health equity, social determinants of health, and disparities in critical
        care outcomes across diverse patient populations and hospital settings. Dataset includes contextual
        factors such as geographic distance to nearest hospital.

        '
    - id: chorus:use:4
      name: Educational and training purposes for AI scientists
      description: 'Training and education of next generation of diverse academic and community AI scientists
        through hands-on experience with real-world critical care datasets. Integrated with AIM-AHEAD Bridge2AI
        for Clinical Care Training Program (Cohorts 1 and 2).

        '

✓ 1/1 R10 3.Data Reuse and Interoperability Use Guidance Provided (intended, prohibited uses)
evidence intended_uses with 4 entries (AI/ML model development for critical care, external validation, health equity research, educational/training); discouraged_uses with 3 entries (clinical decision-making without validation/regulatory approval, re-identification attempts, use without awareness of ongoing data collection limitations); prohibited_uses with 2 entries (re-identification of de-identified patients, redistribution outside DUA); existing_uses with 2 entries (AIM-AHEAD training, peer-reviewed publications); future_use_impacts with 2 entries (transform critical care AI research, risk of algorithmic bias propagation)
quality Comprehensive use guidance covering intended research uses, clear prohibitions (re-identification, redistribution), discouraged practices (premature clinical deployment), existing uses (training programs, publications), and future impact considerations (bias propagation risks)
semantic Use guidance semantically appropriate for research dataset: encourages AI/ML development while prohibiting clinical deployment without validation; addresses ethical concerns (bias propagation, re-identification)
discouraged_uses

    discouraged_uses:
    - id: chorus:discouraged:1
      name: Clinical decision-making without proper validation and regulatory approval
      description: 'Dataset is for research purposes only. AI/ML models developed should undergo appropriate
        clinical validation, regulatory approval, and institutional review before use in patient care or clinical
        decision-making. Models trained on CHoRUS data must undergo independent clinical validation and regulatory
        approval processes (e.g., FDA clearance) before clinical deployment.

        '
    - id: chorus:discouraged:2
      name: Re-identification attempts
      description: 'Attempts to re-identify patients from de-identified data violate ethical principles, data
        use agreements, and legal frameworks established for privacy protection under HIPAA and institutional
        requirements. Re-identification attempts are prohibited by the signed data use agreement.

        '
    - id: chorus:discouraged:3
      name: Use without awareness of ongoing data collection limitations
      description: 'As data collection continues through November 2026 and quality assurance processes are
        ongoing, early dataset versions should be used with awareness of completeness limitations and ongoing
        expansion (from 45K to target 100K+ admissions). EEG extraction and full imaging de-identification
        still in process as of 2025.

        '

✓ 1/1 R10 3.Data Reuse and Interoperability Use Guidance Provided (intended, prohibited uses)
evidence intended_uses with 4 entries (AI/ML model development for critical care, external validation, health equity research, educational/training); discouraged_uses with 3 entries (clinical decision-making without validation/regulatory approval, re-identification attempts, use without awareness of ongoing data collection limitations); prohibited_uses with 2 entries (re-identification of de-identified patients, redistribution outside DUA); existing_uses with 2 entries (AIM-AHEAD training, peer-reviewed publications); future_use_impacts with 2 entries (transform critical care AI research, risk of algorithmic bias propagation)
quality Comprehensive use guidance covering intended research uses, clear prohibitions (re-identification, redistribution), discouraged practices (premature clinical deployment), existing uses (training programs, publications), and future impact considerations (bias propagation risks)
semantic Use guidance semantically appropriate for research dataset: encourages AI/ML development while prohibiting clinical deployment without validation; addresses ethical concerns (bias propagation, re-identification)
prohibited_uses

    prohibited_uses:
    - id: chorus:prohibited:1
      name: Re-identification of de-identified patients
      description: 'Explicitly prohibited under the data use agreement and applicable law (HIPAA). Any attempt
        to re-identify individual patients from the de-identified data is a violation of the data use agreement
        and may result in termination of access and legal consequences.

        '
    - id: chorus:prohibited:2
      name: Redistribution of data outside the data use agreement
      description: 'Redistribution, sharing, or transfer of CHoRUS data to third parties outside the terms
        of the signed data use agreement is explicitly prohibited. Data must remain within the secure enclave
        environment or as explicitly permitted by the agreement.

        '

✓ 1/1 R10 3.Data Reuse and Interoperability Use Guidance Provided (intended, prohibited uses)
evidence intended_uses with 4 entries (AI/ML model development for critical care, external validation, health equity research, educational/training); discouraged_uses with 3 entries (clinical decision-making without validation/regulatory approval, re-identification attempts, use without awareness of ongoing data collection limitations); prohibited_uses with 2 entries (re-identification of de-identified patients, redistribution outside DUA); existing_uses with 2 entries (AIM-AHEAD training, peer-reviewed publications); future_use_impacts with 2 entries (transform critical care AI research, risk of algorithmic bias propagation)
quality Comprehensive use guidance covering intended research uses, clear prohibitions (re-identification, redistribution), discouraged practices (premature clinical deployment), existing uses (training programs, publications), and future impact considerations (bias propagation risks)
semantic Use guidance semantically appropriate for research dataset: encourages AI/ML development while prohibiting clinical deployment without validation; addresses ethical concerns (bias propagation, re-identification)
4/5 R20 Q18 (FAIRness & Accessibility) Reusability (License Clarity)
level License explicitly defines reuse terms with contact details
evidence license_and_use_terms: controlled access license requiring institutional email + signed agreement, prohibits re-identification and redistribution, research purposes only. ip_restrictions: NIH grant terms (OT2OD032701), clinical data ownership with contributing institutions. prohibited_uses: re-identification prohibited, redistribution outside DUA prohibited.
quality License clearly defined with explicit reuse restrictions: research purposes only, no re-identification, no redistribution. Permitted uses implied through intended_uses (AI/ML development, validation, health equity research, education). Minor deduction for no explicit permitted use enumeration in license field itself.
correctness controlled_access_appropriate: correct for sensitive de-identified clinical data; prohibition_specificity: re-identification and redistribution prohibitions are standard for health data
consistency license_aligns_with_intended_uses: research purposes matches intended_uses (AI/ML development, education); license_restrictions_match_prohibited_uses: DUA prohibitions match prohibited_uses section; ip_ownership_clear: institutional ownership with NIH grant terms
existing_uses

    existing_uses:
    - id: chorus:existing:1
      name: AIM-AHEAD Bridge2AI Clinical Care Training Program
      description: 'Subsets of the dataset used in the AIM-AHEAD Bridge2AI for Clinical Care Training Program
        (Cohort 1: 2024-2025, Cohort 2: 2025-2026). Training provides foundational hands-on experience using
        Jupyter Notebooks with Bridge2AI CHoRUS ecosystem, workshops on OHDSI/OMOP common data model and clinical
        AI, and development of practical use cases.

        '
    - id: chorus:existing:2
      name: Peer-reviewed research publications
      description: 'Dataset methodology and design documented in peer-reviewed publications including Neurocritical
        Care journal (doi: 10.1007/s12028-024-02007). Dataset used for initial characterization and training
        activities as the full cohort undergoes expansion and quality assurance.

        '

✓ 1/1 R10 10.Cross-Platform and Community Integration Outreach Materials and Documentation Links
evidence external_resources with 8 entries: CHoRUS project website, GitHub organization (28 repos including software, documentation, SOPs, tooling at https://github.com/chorus-ai), Chorus_SOP documentation site with interactive workflow diagrams (https://github.com/chorus-ai/Chorus_SOP), NIH RePORTER project details, Bridge2AI program page, AIM-AHEAD training program (https://aim-ahead.net/), OHDSI community (https://www.ohdsi.org/), peer-reviewed publication (https://doi.org/10.1007/s12028-024-02007); existing_uses mention AIM-AHEAD training with Jupyter Notebooks and OHDSI/OMOP workshops
quality Extensive outreach and documentation: project website, GitHub organization with 28 repositories, standard operating procedure documentation with interactive diagrams, training program partnerships (AIM-AHEAD with Jupyter Notebooks and workshops), published methodology paper; documentation covers technical implementation (SOPs, GitHub repos), educational materials (training program, workshops), and scientific context (publication)
semantic Outreach materials semantically comprehensive: technical documentation (GitHub, SOPs) for developers, training materials (AIM-AHEAD, Jupyter Notebooks) for users, scientific publications for researchers, community links (OHDSI) for ecosystem integration
✓ 1/1 R10 3.Data Reuse and Interoperability Use Guidance Provided (intended, prohibited uses)
evidence intended_uses with 4 entries (AI/ML model development for critical care, external validation, health equity research, educational/training); discouraged_uses with 3 entries (clinical decision-making without validation/regulatory approval, re-identification attempts, use without awareness of ongoing data collection limitations); prohibited_uses with 2 entries (re-identification of de-identified patients, redistribution outside DUA); existing_uses with 2 entries (AIM-AHEAD training, peer-reviewed publications); future_use_impacts with 2 entries (transform critical care AI research, risk of algorithmic bias propagation)
quality Comprehensive use guidance covering intended research uses, clear prohibitions (re-identification, redistribution), discouraged practices (premature clinical deployment), existing uses (training programs, publications), and future impact considerations (bias propagation risks)
semantic Use guidance semantically appropriate for research dataset: encourages AI/ML development while prohibiting clinical deployment without validation; addresses ethical concerns (bias propagation, re-identification)
future_use_impacts

    future_use_impacts:
    - id: chorus:impact:1
      name: Potential to transform critical care AI research
      description: 'The CHoRUS dataset has potential to become a foundational resource for critical care AI/ML
        research, enabling development of models that could improve patient outcomes in ICU settings across
        diverse populations. External validation capabilities support broader adoption of AI-developed clinical
        tools.

        '
    - id: chorus:impact:2
      name: Risk of algorithmic bias propagation
      description: 'AI/ML models trained on CHoRUS data may inherit biases present in clinical documentation,
        treatment patterns, or data collection at academic medical centers. Models developed without appropriate
        bias mitigation could perpetuate or amplify healthcare disparities if deployed in clinical practice
        without careful validation.

        '

✓ 1/1 R10 3.Data Reuse and Interoperability Use Guidance Provided (intended, prohibited uses)
evidence intended_uses with 4 entries (AI/ML model development for critical care, external validation, health equity research, educational/training); discouraged_uses with 3 entries (clinical decision-making without validation/regulatory approval, re-identification attempts, use without awareness of ongoing data collection limitations); prohibited_uses with 2 entries (re-identification of de-identified patients, redistribution outside DUA); existing_uses with 2 entries (AIM-AHEAD training, peer-reviewed publications); future_use_impacts with 2 entries (transform critical care AI research, risk of algorithmic bias propagation)
quality Comprehensive use guidance covering intended research uses, clear prohibitions (re-identification, redistribution), discouraged practices (premature clinical deployment), existing uses (training programs, publications), and future impact considerations (bias propagation risks)
semantic Use guidance semantically appropriate for research dataset: encourages AI/ML development while prohibiting clinical deployment without validation; addresses ethical concerns (bias propagation, re-identification)
distribution_formats

    distribution_formats:
    - id: chorus:format:1
      name: OMOP Common Data Model (structured EHR data)
      description: 'Structured electronic health record data distributed in OMOP Common Data Model format
        enabling use of OHDSI tool stack for analysis. Contains 1.6 billion rows. Available in secure enclave
        with controlled access. Published OMOP schema metadata available.

        '
    - id: chorus:format:2
      name: WFDB waveform format (bedside monitor telemetry)
      description: 'Waveform telemetry data (23 TB) distributed in WFDB (WaveForm DataBase) format following
        extended PhysioNet schema. Available in secure enclave with controlled access. Published PhysioNet
        schema (extended) metadata available.

        '
    - id: chorus:format:3
      name: DICOM format (medical imaging)
      description: 'Medical imaging data (7,642 admissions with radiology; 1,000 images currently available)
        distributed in DICOM format with comprehensive metadata following DICOM schema. De-identification
        in process for larger cohort. Planned controlled access.

        '
    - id: chorus:format:4
      name: OHNLP tokenized format (clinical notes)
      description: 'Clinical notes distributed as OHNLP-tokenized text following open source OHNLP schema.
        Stored locally at contributing sites (except tokens). Controlled access planned.

        '
    - id: chorus:format:5
      name: EDF+ and Persyst formats (EEG waveforms)
      description: 'EEG waveform data distributed in EDF+ (European Data Format) and Persyst formats following
        open source schemas. Extraction in process, planned for controlled access.

        '

⚠ low R20 · schema_limitation
issue Core schema may lack DOI, conforms_to, conforms_to_schema fields
fields id, distribution_formats
fix Verify if D4D-core schema includes DOI and schema conformance fields. Standards mentioned narratively (OMOP, DICOM, WFDB, OHNLP) but no formal conforms_to references.
✓ 1/1 R10 2.Dataset Access and Retrieval Distribution Formats and File Types Specified
evidence distribution_formats with 5 entries: OMOP CDM (structured EHR), WFDB (waveform telemetry, 23TB), DICOM (medical imaging), OHNLP tokenized (clinical notes, TXT/text/plain), EDF+/Persyst (EEG); distributions[3] specifies format: TXT, media_type: text/plain for clinical notes
quality Comprehensive format documentation across all data modalities with specific standards (OMOP, WFDB, DICOM, OHNLP, EDF+/Persyst); clinical notes include media_type
semantic Formats semantically appropriate for clinical data types: OMOP for structured EHR, WFDB for physiological waveforms, DICOM for imaging, OHNLP for NLP-ready text, EDF+ for EEG
✓ 1/1 R10 3.Data Reuse and Interoperability Data Formats Are Standardized (encoding, format)
evidence distribution_formats: OMOP Common Data Model (structured EHR), WFDB (waveform telemetry), DICOM (medical imaging), OHNLP tokenized (clinical notes), EDF+/Persyst (EEG); distributions[3] specifies format: TXT, media_type: text/plain; language: en
quality All data modalities use community-standard formats: OMOP CDM (OHDSI standard), WFDB (PhysioNet standard), DICOM (medical imaging standard), OHNLP (open source NLP), EDF+ (European Data Format for EEG); language specified as English
semantic Formats are internationally recognized standards enabling interoperability; OMOP enables OHDSI tool stack, DICOM enables medical imaging workflows, WFDB enables PhysioNet tools
✗ 0/1 R10 3.Data Reuse and Interoperability Schema or Ontology Conformance Stated
evidence No conforms_to or conforms_to_schema fields in D4D-core; preprocessing_strategies and distribution_formats describe OMOP, DICOM, WFDB, OHNLP, EDF+/Persyst schemas but not in dedicated conformance field
quality D4D-core schema lacks conforms_to field; schema conformance information present in descriptive text (OMOP CDM, DICOM schema, PhysioNet WFDB schema, OHNLP schema, EDF+ schema) but not structured metadata
semantic Full D4D schema includes conforms_to field; D4D-core subset does not expose this structured metadata
✓ 1/1 R10 5.Data Composition and Structure Variable-Level Metadata and Tabular Flag
evidence is_tabular: false; distribution_formats describe multi-modal data (OMOP CDM structured EHR, WFDB waveforms, DICOM imaging, OHNLP tokenized notes, EDF+/Persyst EEG); no variables field in D4D-core
quality Tabular flag correctly set to false for multi-modal dataset; D4D-core lacks variables field but distribution_formats and preprocessing_strategies describe each modality's structure; OMOP CDM schema metadata published externally (referenced in external_resources)
semantic is_tabular=false semantically correct for dataset with multiple distinct formats (structured EHR, waveforms, imaging, text, EEG); variable metadata delegated to modality-specific schemas (OMOP CDM tables, DICOM tags, WFDB headers)
1/5 R20 Q10 (Metadata Quality & Content) Interoperability and Standardization
level Standard formats mentioned but no schema conformance fields
evidence distribution_formats: OMOP Common Data Model, DICOM, WFDB, OHNLP, EDF+, Persyst (all industry standards). preprocessing_strategies mention validated semantic mappings and schema compliance validation. No conforms_to or conforms_to_schema fields in core schema.
quality Uses multiple recognized industry standards (OMOP, DICOM, WFDB, OHNLP, EDF+) but core schema lacks explicit conforms_to/conforms_to_schema fields for formal schema references. Standards mentioned narratively.
correctness format_standards: OMOP, DICOM, WFDB, OHNLP, EDF+ are all recognized healthcare/clinical data standards; omop_description: correctly describes OMOP Common Data Model and OHDSI tool stack; dicom_description: correctly identifies DICOM as imaging standard; wfdb_description: correctly identifies WFDB as PhysioNet waveform format
consistency formats_match_data_types: OMOP for EHR, DICOM for imaging, WFDB for waveforms, OHNLP for notes (all appropriate); schema_validation_mentioned: preprocessing_strategies describe schema compliance validation; note: Core schema may not include conforms_to/conforms_to_schema fields
4/5 R20 Q17 (FAIRness & Accessibility) Accessibility (Access Mechanism)
level Fully defined access path
evidence license_and_use_terms: institutional email registration required, signed licensing agreement, access request contacts (dbold@emory.edu, jared.houghtaling@tuftsmedicine.org). distribution_formats describe controlled access via secure enclave for each data type. distributions specify access at https://chorus4ai.org/ with controlled access requirements.
quality Very clear access mechanism: institutional email registration → signed licensing agreement → secure enclave access. Contact emails provided. Specific requirements documented. Minor deduction for no download vs. enclave-only clarification.
correctness email_format: dbold@emory.edu and jared.houghtaling@tuftsmedicine.org are valid email formats; institutional_email_requirement: appropriate for controlled access to sensitive health data; secure_enclave_mention: Azure-based infrastructure mentioned elsewhere
consistency access_requirements_match_license: institutional email and DUA consistent with controlled access license; all_distributions_same_mechanism: all 5 distributions use same controlled access pattern; deidentified_but_controlled: correctly maintains controlled access despite de-identification
distribution_dates

    distribution_dates:
    - id: chorus:distdate:1
      name: Initial release date
      description: 'Dataset made available for access beginning September 2022 (project start date). Current
        release (August 2025) includes 50,000 patient admissions with 1.6 billion rows of EHR OMOP data. Ongoing
        expansion through November 30, 2026 project end date.

        '

no field-level feedback matched
maintainers

    maintainers:
    - id: chorus:maintainer:1
      name: CHoRUS Consortium
      description: 'Multi-institutional consortium managing dataset maintenance including 14 data acquisition
        centers, coordinating teams at Massachusetts General Hospital (lead), University of Florida, UT Health
        Science Center, and Tufts Medicine, plus infrastructure development team. Standards, Data Acquisition,
        and Tooling sub-teams manage ongoing data delivery and quality. Contact: cmccrary@mgh.harvard.edu
        (Ciera McCrary, Program Manager, MGH).

        '
    - id: chorus:maintainer:2
      name: CHoRUS GitHub Organization (chorus-ai)
      description: 'Active GitHub organization housing repositories for software, semantic mappings, standard
        operating protocols, and project management. 28 repositories with comprehensive documentation and
        community support. GitHub organization at https://github.com/chorus-ai.

        '

no field-level feedback matched
updates

    updates:
      id: chorus:updates:1
      name: Ongoing data collection and continuous expansion
      description: 'Dataset updated continuously as data collection progresses at 14 acquisition centers.
        As of August 2025, covers 14 hospitals with over 45,000 unique admissions; current released dataset
        includes 50,000 patient admissions (ICU, PICU, NICU) and 1.6 billion rows of EHR OMOP data. Target
        exceeds 100,000 critically ill patients. Project timeline extends through November 30, 2026 (approved
        no-cost extension). Regular status updates tracked through GitHub project management system.

        '

⚠ low R20 · schema_limitation
issue Core schema may lack version, errata, release_notes, citation fields
fields version_access, updates
fix Version access mechanism described but no explicit version number. Publication listed in external_resources but no dedicated citation field. Verify core schema structure.
✗ 0/1 R10 6.Data Provenance and Version Tracking Dataset Version Number Provided
evidence No version field in D4D-core schema; updates describe 'ongoing data collection and continuous expansion' with temporal snapshots (Aug 2025: 45K admissions, 50K released; target 100K+ by Nov 2026)
quality D4D-core schema lacks version field; updates section provides temporal context but no formal version number
semantic Versioning information present in narrative form (Aug 2025 snapshot) but not structured as semantic version number
✗ 0/1 R10 6.Data Provenance and Version Tracking Change Descriptions and Errata Provided
evidence No errata field in D4D-core schema; updates describe ongoing expansion (45K → 100K+ admissions) and processing status (EEG extraction in process, imaging de-id in process) but no structured change log or errata documentation
quality D4D-core schema lacks errata field; updates section describes ongoing expansion but not version-to-version changes or corrections
semantic Change documentation present in narrative form (ongoing collection through Nov 2026) but not structured errata or version release notes
✓ 1/1 R10 6.Data Provenance and Version Tracking Update Schedule or Frequency Indicated
evidence updates: Dataset updated continuously as data collection progresses at 14 acquisition centers; project timeline extends through November 30, 2026 (approved no-cost extension); regular status updates tracked through GitHub project management system; collection_timeframes: September 1, 2022 through November 30, 2026
quality Update schedule documented: continuous updates during ongoing collection (Sept 2022 - Nov 2026); regular status tracking via GitHub; temporal milestones provided (current Aug 2025 snapshot, end date Nov 2026)
semantic Update schedule semantically clear: continuous expansion during project period with regular status tracking; end date (Nov 2026) provides timeline boundary
2/5 R20 Q13 (Technical Documentation) Version History Documentation
level Updates and version access documented but no version number or errata
evidence updates: ongoing data collection, current 50,000 admissions expanding to 100K+ target, August 2025 status. version_access: controlled access versioned releases through secure enclave. No version field, errata, or release_notes in core schema.
quality Update plans and version access mechanism described but lacking explicit version number, errata documentation, or structured release notes. Core schema may not include version, errata, or release_notes fields.
correctness update_timeline: August 2025 status with Nov 2026 completion is plausible; version_access_mechanism: secure enclave with versioning is appropriate for controlled access data
consistency update_timeline_matches_project_dates: ongoing through Nov 2026 consistent with funding period; version_growth: 45K to 50K to 100K+ progression is logical; note: Core schema may lack version, errata, release_notes fields
0/5 R20 Q19 (FAIRness & Accessibility) Data Integrity and Provenance
level Limited provenance metadata
evidence updates: describes ongoing expansion (45K to 50K to 100K+) with August 2025 status. version_access: mentions version history maintenance. No structured change logs, timestamps, or detailed version tracking in core schema.
quality Update plans described but lacking structured change logs, version timestamps, or formal provenance tracking. Core schema may not support detailed provenance fields. General update narrative provided but not formal version history.
correctness update_description: ongoing expansion narrative is plausible; status_date: August 2025 status date is plausible
consistency update_timeline_matches_project: expansion through Nov 2026 matches funding period; note: Core schema likely lacks detailed provenance/changelog fields
retention_limit

    retention_limit:
      id: chorus:retention:1
      name: Long-term dataset retention per NIH data sharing policies
      description: 'Digital data maintained according to NIH data sharing policies and institutional requirements
        at participating centers. Controlled access model ensures long-term availability for research while
        protecting patient privacy. Funded under NIH award OT2OD032701 with project end date of November 30,
        2026.

        '

no field-level feedback matched
version_access

    version_access:
      id: chorus:versionaccess:1
      name: Controlled access versioned releases
      description: 'Dataset versions accessible through secure enclave with controlled access. Users must
        register with institutional email and sign licensing agreement to access current and future versions.
        Version history maintained as data collection expands from 45K to target 100K+ critically ill patient
        admissions.

        '

⚠ low R20 · schema_limitation
issue Core schema may lack version, errata, release_notes, citation fields
fields version_access, updates
fix Version access mechanism described but no explicit version number. Publication listed in external_resources but no dedicated citation field. Verify core schema structure.
✓ 1/1 R10 6.Data Provenance and Version Tracking Version Access Methods Documented
evidence version_access: Dataset versions accessible through secure enclave with controlled access; users must register with institutional email and sign licensing agreement to access current and future versions; version history maintained as data collection expands from 45K to target 100K+ admissions
quality Version access method documented: controlled access enclave with registration/DUA for current and future versions; version history maintained during ongoing expansion
semantic Version access semantically appropriate for controlled access dataset; users receive access to evolving dataset through secure enclave
2/5 R20 Q13 (Technical Documentation) Version History Documentation
level Updates and version access documented but no version number or errata
evidence updates: ongoing data collection, current 50,000 admissions expanding to 100K+ target, August 2025 status. version_access: controlled access versioned releases through secure enclave. No version field, errata, or release_notes in core schema.
quality Update plans and version access mechanism described but lacking explicit version number, errata documentation, or structured release notes. Core schema may not include version, errata, or release_notes fields.
correctness update_timeline: August 2025 status with Nov 2026 completion is plausible; version_access_mechanism: secure enclave with versioning is appropriate for controlled access data
consistency update_timeline_matches_project_dates: ongoing through Nov 2026 consistent with funding period; version_growth: 45K to 50K to 100K+ progression is logical; note: Core schema may lack version, errata, release_notes fields
0/5 R20 Q19 (FAIRness & Accessibility) Data Integrity and Provenance
level Limited provenance metadata
evidence updates: describes ongoing expansion (45K to 50K to 100K+) with August 2025 status. version_access: mentions version history maintenance. No structured change logs, timestamps, or detailed version tracking in core schema.
quality Update plans described but lacking structured change logs, version timestamps, or formal provenance tracking. Core schema may not support detailed provenance fields. General update narrative provided but not formal version history.
correctness update_description: ongoing expansion narrative is plausible; status_date: August 2025 status date is plausible
consistency update_timeline_matches_project: expansion through Nov 2026 matches funding period; note: Core schema likely lacks detailed provenance/changelog fields
extension_mechanism

    extension_mechanism:
      id: chorus:extension:1
      name: Site contribution and GitHub-based development
      description: 'Dataset extended through contributions from 14 data acquisition centers via standardized
        extraction protocols (Chorus_SOP). Software tools and semantic mappings extended through GitHub organization
        (chorus-ai) with 28 active repositories. New sites may join through established data acquisition framework.
        Community contributions to mapping efforts via chorus-mapping repository and clinical validation SOP.

        '

no field-level feedback matched
ethical_reviews

    ethical_reviews:
    - id: chorus:ethics:1
      name: Institutional Review Board approvals at 14 acquisition centers
      description: 'Retrospective data collection approved through institutional review board processes at
        all 14 data acquisition centers across the United States. Community-facing ethics focus groups conducted
        to determine what data is appropriate for public sharing. Legal framework established for collecting
        data at scale.

        '

⚠ low R10 · consistency
issue human_subject_research.involves_human_subjects=true with ethical_reviews present but lacks detailed IRB institution names
fields human_subject_research, ethical_reviews
fix Add specific IRB institution names for each of the 14 acquisition centers
✓ 1/1 R10 4.Ethical Use and Privacy Safeguards IRB or Ethics Review Documented
evidence ethical_reviews: IRB approvals at all 14 data acquisition centers, community-facing ethics focus groups conducted, legal framework established for collecting data at scale; human_subject_research.involves_human_subjects: true with retrospective data collection from critically ill patients
quality IRB approvals documented for all 14 acquisition centers; includes community ethics engagement (focus groups); consistency check passed (involves_human_subjects=true AND ethical_reviews present)
semantic Ethical review appropriate for multi-institutional retrospective clinical data collection; community focus groups demonstrate patient-centered ethics approach
✗ 0/1 R10 9.Dataset Evaluation and Limitations Disclosure Ethical Review Details Including Conflicts
evidence ethical_reviews: IRB approvals at all 14 acquisition centers, community ethics focus groups, legal framework established; human_subject_research describes patient-focused ethics pillar, multi-disciplinary expertise (law, ethics, health services, biomedical science, engineering, scientific publications), AIM-AHEAD ethics curriculum (safety, risk, legal considerations, IRB, HIPAA/GDPR compliance); no conflicts of interest documentation in D4D-core schema
quality Ethical review process documented (IRB approvals at 14 sites, community focus groups, multi-disciplinary ethics expertise); D4D-core schema lacks conflicts_of_interest or financial_disclosures fields; full ethical review details present but no conflict-of-interest disclosure
semantic Ethical review documentation semantically appropriate for multi-institutional clinical research; lacks explicit conflict-of-interest disclosure which would be expected for comprehensive ethical transparency
4/5 R20 Q8 (Metadata Quality & Content) Ethical and Privacy Declarations
level Very comprehensive ethics documentation
evidence ethical_reviews: IRB approvals at all 14 data acquisition centers, community-facing ethics focus groups. human_subject_research: retrospective critically ill patients (ICU, PICU, NICU), involves_human_subjects=true. informed_consent: waiver of consent framework with community ethics focus groups. is_deidentified: de-identified via OMOP transformation, privacy scanning tools, OHNLP tokenization. at_risk_populations: critically ill patients, pediatric (PICU), neonatal (NICU) vulnerable populations. No participant_compensation or participant_privacy fields in core schema.
quality Comprehensive ethics documentation covering IRB approvals, vulnerable populations, deidentification methods, and informed consent approach. Core schema may lack participant_compensation and explicit participant_privacy fields, but privacy protections described in is_deidentified.
correctness irb_plausibility: 14 acquisition centers requiring 14 IRB approvals is correct multi-site approach; deidentification_methods: OMOP transformation, privacy scanning, OHNLP tokenization appropriate for clinical data; vulnerable_population_identification: correctly identifies critically ill, pediatric, neonatal as vulnerable
consistency human_subjects_flag_matches_ethics: True; irb_count_matches_site_count: 14 acquisition centers = 14 IRB approvals (consistent); deidentification_aligns_with_data_types: different methods for EHR (OMOP), notes (OHNLP), imaging (DICOM header removal); consent_waiver_appropriate_for_retrospective: True
human_subject_research

    human_subject_research:
      id: chorus:hsr:1
      name: CHoRUS Human Subjects Research
      description: 'Retrospective data collection from critically ill patients (ICU, PICU, NICU) approved
        through institutional review processes at 14 data acquisition centers. Community-facing ethics focus
        groups conducted to determine what data is appropriate for public sharing. Legal framework established
        for collecting data at scale. Patient-focused efforts determine ethical and legal approaches to manage
        privacy and bias while accounting for Social Determinants of Health. Project includes expertise from
        law, ethics, health services, biomedical science, engineering, and scientific journal publications
        disciplines. Ethics of AI component addressed through AIM-AHEAD training curriculum (safety, risk,
        and legal considerations; IRB, HIPAA/GDPR compliance for OMOP/FHIR data).

        '
      involves_human_subjects: true

⚠ low R10 · consistency
issue human_subject_research.involves_human_subjects=true with ethical_reviews present but lacks detailed IRB institution names
fields human_subject_research, ethical_reviews
fix Add specific IRB institution names for each of the 14 acquisition centers
✓ 1/1 R10 4.Ethical Use and Privacy Safeguards IRB or Ethics Review Documented
evidence ethical_reviews: IRB approvals at all 14 data acquisition centers, community-facing ethics focus groups conducted, legal framework established for collecting data at scale; human_subject_research.involves_human_subjects: true with retrospective data collection from critically ill patients
quality IRB approvals documented for all 14 acquisition centers; includes community ethics engagement (focus groups); consistency check passed (involves_human_subjects=true AND ethical_reviews present)
semantic Ethical review appropriate for multi-institutional retrospective clinical data collection; community focus groups demonstrate patient-centered ethics approach
✗ 0/1 R10 9.Dataset Evaluation and Limitations Disclosure Ethical Review Details Including Conflicts
evidence ethical_reviews: IRB approvals at all 14 acquisition centers, community ethics focus groups, legal framework established; human_subject_research describes patient-focused ethics pillar, multi-disciplinary expertise (law, ethics, health services, biomedical science, engineering, scientific publications), AIM-AHEAD ethics curriculum (safety, risk, legal considerations, IRB, HIPAA/GDPR compliance); no conflicts of interest documentation in D4D-core schema
quality Ethical review process documented (IRB approvals at 14 sites, community focus groups, multi-disciplinary ethics expertise); D4D-core schema lacks conflicts_of_interest or financial_disclosures fields; full ethical review details present but no conflict-of-interest disclosure
semantic Ethical review documentation semantically appropriate for multi-institutional clinical research; lacks explicit conflict-of-interest disclosure which would be expected for comprehensive ethical transparency
4/5 R20 Q8 (Metadata Quality & Content) Ethical and Privacy Declarations
level Very comprehensive ethics documentation
evidence ethical_reviews: IRB approvals at all 14 data acquisition centers, community-facing ethics focus groups. human_subject_research: retrospective critically ill patients (ICU, PICU, NICU), involves_human_subjects=true. informed_consent: waiver of consent framework with community ethics focus groups. is_deidentified: de-identified via OMOP transformation, privacy scanning tools, OHNLP tokenization. at_risk_populations: critically ill patients, pediatric (PICU), neonatal (NICU) vulnerable populations. No participant_compensation or participant_privacy fields in core schema.
quality Comprehensive ethics documentation covering IRB approvals, vulnerable populations, deidentification methods, and informed consent approach. Core schema may lack participant_compensation and explicit participant_privacy fields, but privacy protections described in is_deidentified.
correctness irb_plausibility: 14 acquisition centers requiring 14 IRB approvals is correct multi-site approach; deidentification_methods: OMOP transformation, privacy scanning, OHNLP tokenization appropriate for clinical data; vulnerable_population_identification: correctly identifies critically ill, pediatric, neonatal as vulnerable
consistency human_subjects_flag_matches_ethics: True; irb_count_matches_site_count: 14 acquisition centers = 14 IRB approvals (consistent); deidentification_aligns_with_data_types: different methods for EHR (OMOP), notes (OHNLP), imaging (DICOM header removal); consent_waiver_appropriate_for_retrospective: True
informed_consent

    informed_consent:
    - id: chorus:consent:1
      name: Retrospective data and waiver of consent framework
      description: 'Retrospective data collection from critically ill patients using waiver of individual
        consent framework approved by IRBs at participating institutions. Community-facing ethics focus groups
        engaged to determine appropriate data for public sharing and guide ethical data governance. Patient-focused
        ethics pillar of CHoRUS project determines ethical and legal approaches to privacy, bias, and Social
        Determinants of Health considerations.

        '

✓ 1/1 R10 4.Ethical Use and Privacy Safeguards Informed Consent Obtained from Participants
evidence informed_consent: Retrospective data collection using waiver of individual consent framework approved by IRBs at participating institutions; community-facing ethics focus groups engaged to determine appropriate data for public sharing; patient-focused ethics pillar determines ethical approaches to privacy, bias, and Social Determinants of Health
quality Consent framework documented: waiver of individual consent for retrospective data collection (IRB-approved); community ethics engagement as proxy for patient input; patient-focused ethics pillar guides privacy and bias considerations
semantic Waiver of consent appropriate for retrospective clinical data research under 45 CFR 46.116(d); community focus groups provide ethical governance for public data sharing decisions
4/5 R20 Q8 (Metadata Quality & Content) Ethical and Privacy Declarations
level Very comprehensive ethics documentation
evidence ethical_reviews: IRB approvals at all 14 data acquisition centers, community-facing ethics focus groups. human_subject_research: retrospective critically ill patients (ICU, PICU, NICU), involves_human_subjects=true. informed_consent: waiver of consent framework with community ethics focus groups. is_deidentified: de-identified via OMOP transformation, privacy scanning tools, OHNLP tokenization. at_risk_populations: critically ill patients, pediatric (PICU), neonatal (NICU) vulnerable populations. No participant_compensation or participant_privacy fields in core schema.
quality Comprehensive ethics documentation covering IRB approvals, vulnerable populations, deidentification methods, and informed consent approach. Core schema may lack participant_compensation and explicit participant_privacy fields, but privacy protections described in is_deidentified.
correctness irb_plausibility: 14 acquisition centers requiring 14 IRB approvals is correct multi-site approach; deidentification_methods: OMOP transformation, privacy scanning, OHNLP tokenization appropriate for clinical data; vulnerable_population_identification: correctly identifies critically ill, pediatric, neonatal as vulnerable
consistency human_subjects_flag_matches_ethics: True; irb_count_matches_site_count: 14 acquisition centers = 14 IRB approvals (consistent); deidentification_aligns_with_data_types: different methods for EHR (OMOP), notes (OHNLP), imaging (DICOM header removal); consent_waiver_appropriate_for_retrospective: True
at_risk_populations

    at_risk_populations:
      id: chorus:atrisk:1
      name: Critically ill patients and vulnerable ICU populations
      description: 'Dataset includes critically ill patients (ICU, PICU, NICU) who are vulnerable due to acute
        illness and critical care needs. Pediatric (PICU) and neonatal (NICU) patients represent particularly
        vulnerable populations. Dataset also includes patients with diverse social determinants of health
        and geographic factors. Privacy protections include de-identification, controlled access model, and
        data use agreement restrictions. Community ethics focus groups engaged to protect patient interests.

        '

✗ 0/1 R10 4.Ethical Use and Privacy Safeguards Vulnerable Populations and Compensation Documented
evidence at_risk_populations: Critically ill patients (ICU, PICU, NICU) vulnerable due to acute illness; pediatric and neonatal patients particularly vulnerable; patients with diverse social determinants of health; privacy protections include de-id + controlled access + DUA; community ethics focus groups engaged; no participant_compensation field in D4D-core
quality At-risk populations clearly identified (critically ill ICU/PICU/NICU patients) with appropriate privacy protections; D4D-core schema lacks participant_compensation field; retrospective waiver-of-consent framework means no compensation discussion applicable
semantic Vulnerable population protections appropriate for critically ill patient data; retrospective data collection with waiver of consent means participant compensation not applicable
4/5 R20 Q8 (Metadata Quality & Content) Ethical and Privacy Declarations
level Very comprehensive ethics documentation
evidence ethical_reviews: IRB approvals at all 14 data acquisition centers, community-facing ethics focus groups. human_subject_research: retrospective critically ill patients (ICU, PICU, NICU), involves_human_subjects=true. informed_consent: waiver of consent framework with community ethics focus groups. is_deidentified: de-identified via OMOP transformation, privacy scanning tools, OHNLP tokenization. at_risk_populations: critically ill patients, pediatric (PICU), neonatal (NICU) vulnerable populations. No participant_compensation or participant_privacy fields in core schema.
quality Comprehensive ethics documentation covering IRB approvals, vulnerable populations, deidentification methods, and informed consent approach. Core schema may lack participant_compensation and explicit participant_privacy fields, but privacy protections described in is_deidentified.
correctness irb_plausibility: 14 acquisition centers requiring 14 IRB approvals is correct multi-site approach; deidentification_methods: OMOP transformation, privacy scanning, OHNLP tokenization appropriate for clinical data; vulnerable_population_identification: correctly identifies critically ill, pediatric, neonatal as vulnerable
consistency human_subjects_flag_matches_ethics: True; irb_count_matches_site_count: 14 acquisition centers = 14 IRB approvals (consistent); deidentification_aligns_with_data_types: different methods for EHR (OMOP), notes (OHNLP), imaging (DICOM header removal); consent_waiver_appropriate_for_retrospective: True
license_and_use_terms

    license_and_use_terms:
      id: chorus:license:1
      name: CHoRUS Controlled Access License with Data Use Agreement
      description: 'Dataset distributed under controlled access requiring institutional email registration
        and signed licensing agreement. Access granted after review and approval process. Participants must
        complete registration form with name, institutional email (not personal), and institution. Once approved,
        users receive email with access instructions to CHoRUS secure enclave. For training program access,
        program administrators assist with licensing. Access request contacts - dbold@emory.edu or jared.houghtaling@tuftsmedicine.org.
        Funded under NIH award OT2OD032701.

        '

✓ 1/1 R10 2.Dataset Access and Retrieval Access Policy and IP Restrictions Defined
evidence license_and_use_terms: Controlled Access License with Data Use Agreement requiring institutional email registration and signed licensing agreement; ip_restrictions: Data use governed by CHoRUS licensing agreement and NIH grant terms (OT2OD032701), GitHub repos use MIT/Apache-2.0 licenses
quality Clear access policy with controlled access model, DUA requirements, institutional email verification, and approval process; IP governed by licensing agreement
semantic Access policy semantically appropriate for sensitive clinical data; controlled access with DUA is standard practice for de-identified health data
✓ 1/1 R10 3.Data Reuse and Interoperability License Terms Allow Reuse
evidence license: Controlled Access - Data Use Agreement Required (OT2OD032701); license_and_use_terms: CHoRUS Controlled Access License with DUA requiring institutional email registration, signed agreement, review/approval process
quality Clear license terms allowing reuse for research purposes under controlled access model; DUA process documented with contact information (dbold@emory.edu, jared.houghtaling@tuftsmedicine.org)
semantic Controlled access with DUA is appropriate reuse model for de-identified clinical data; balances data sharing with privacy protection
4/5 R20 Q17 (FAIRness & Accessibility) Accessibility (Access Mechanism)
level Fully defined access path
evidence license_and_use_terms: institutional email registration required, signed licensing agreement, access request contacts (dbold@emory.edu, jared.houghtaling@tuftsmedicine.org). distribution_formats describe controlled access via secure enclave for each data type. distributions specify access at https://chorus4ai.org/ with controlled access requirements.
quality Very clear access mechanism: institutional email registration → signed licensing agreement → secure enclave access. Contact emails provided. Specific requirements documented. Minor deduction for no download vs. enclave-only clarification.
correctness email_format: dbold@emory.edu and jared.houghtaling@tuftsmedicine.org are valid email formats; institutional_email_requirement: appropriate for controlled access to sensitive health data; secure_enclave_mention: Azure-based infrastructure mentioned elsewhere
consistency access_requirements_match_license: institutional email and DUA consistent with controlled access license; all_distributions_same_mechanism: all 5 distributions use same controlled access pattern; deidentified_but_controlled: correctly maintains controlled access despite de-identification
4/5 R20 Q18 (FAIRness & Accessibility) Reusability (License Clarity)
level License explicitly defines reuse terms with contact details
evidence license_and_use_terms: controlled access license requiring institutional email + signed agreement, prohibits re-identification and redistribution, research purposes only. ip_restrictions: NIH grant terms (OT2OD032701), clinical data ownership with contributing institutions. prohibited_uses: re-identification prohibited, redistribution outside DUA prohibited.
quality License clearly defined with explicit reuse restrictions: research purposes only, no re-identification, no redistribution. Permitted uses implied through intended_uses (AI/ML development, validation, health equity research, education). Minor deduction for no explicit permitted use enumeration in license field itself.
correctness controlled_access_appropriate: correct for sensitive de-identified clinical data; prohibition_specificity: re-identification and redistribution prohibitions are standard for health data
consistency license_aligns_with_intended_uses: research purposes matches intended_uses (AI/ML development, education); license_restrictions_match_prohibited_uses: DUA prohibitions match prohibited_uses section; ip_ownership_clear: institutional ownership with NIH grant terms
5/5 R20 Q9 (Metadata Quality & Content) Access Requirements and Governance Documentation
level License + restrictions + confidentiality classification
evidence license_and_use_terms: controlled access requiring institutional email registration and signed licensing agreement, access request contacts provided. ip_restrictions: governed by CHoRUS licensing agreement and NIH grant terms (OT2OD032701), GitHub repos use MIT/Apache-2.0 licenses. regulatory_restrictions: HIPAA compliance, 45 CFR 46 (Common Rule), institutional data privacy regulations, NIH Common Fund ethical AI requirements, export control regulations. No explicit confidentiality_level field in core schema.
quality Excellent governance documentation with clear access policy, licensing requirements, IP framework, and comprehensive regulatory restrictions. Core schema may lack explicit confidentiality_level field, but sensitivity clearly implied by controlled access requirement.
correctness regulatory_citations: HIPAA and 45 CFR 46 are correct US regulations for health data and human subjects; license_types: MIT and Apache-2.0 are valid open source licenses for software; access_mechanism: institutional email + DUA is standard controlled access pattern
consistency controlled_access_aligns_with_sensitivity: True; deidentified_but_controlled: correctly maintains controlled access despite de-identification due to re-id risk; ip_restrictions_match_license: NIH grant terms + institutional ownership consistent with controlled access; regulatory_requirements_match_data_type: HIPAA appropriate for clinical data
ip_restrictions

    ip_restrictions:
      id: chorus:ip:1
      name: Institutional data use agreement and NIH award terms
      description: 'Data use is governed by the CHoRUS licensing agreement and NIH grant terms (OT2OD032701).
        GitHub repositories use MIT License and Apache-2.0 licenses for software components. Clinical data
        ownership remains with contributing institutions subject to applicable law. Re-use outside terms of
        the data use agreement is prohibited.

        '

✓ 1/1 R10 2.Dataset Access and Retrieval Access Policy and IP Restrictions Defined
evidence license_and_use_terms: Controlled Access License with Data Use Agreement requiring institutional email registration and signed licensing agreement; ip_restrictions: Data use governed by CHoRUS licensing agreement and NIH grant terms (OT2OD032701), GitHub repos use MIT/Apache-2.0 licenses
quality Clear access policy with controlled access model, DUA requirements, institutional email verification, and approval process; IP governed by licensing agreement
semantic Access policy semantically appropriate for sensitive clinical data; controlled access with DUA is standard practice for de-identified health data
4/5 R20 Q18 (FAIRness & Accessibility) Reusability (License Clarity)
level License explicitly defines reuse terms with contact details
evidence license_and_use_terms: controlled access license requiring institutional email + signed agreement, prohibits re-identification and redistribution, research purposes only. ip_restrictions: NIH grant terms (OT2OD032701), clinical data ownership with contributing institutions. prohibited_uses: re-identification prohibited, redistribution outside DUA prohibited.
quality License clearly defined with explicit reuse restrictions: research purposes only, no re-identification, no redistribution. Permitted uses implied through intended_uses (AI/ML development, validation, health equity research, education). Minor deduction for no explicit permitted use enumeration in license field itself.
correctness controlled_access_appropriate: correct for sensitive de-identified clinical data; prohibition_specificity: re-identification and redistribution prohibitions are standard for health data
consistency license_aligns_with_intended_uses: research purposes matches intended_uses (AI/ML development, education); license_restrictions_match_prohibited_uses: DUA prohibitions match prohibited_uses section; ip_ownership_clear: institutional ownership with NIH grant terms
5/5 R20 Q9 (Metadata Quality & Content) Access Requirements and Governance Documentation
level License + restrictions + confidentiality classification
evidence license_and_use_terms: controlled access requiring institutional email registration and signed licensing agreement, access request contacts provided. ip_restrictions: governed by CHoRUS licensing agreement and NIH grant terms (OT2OD032701), GitHub repos use MIT/Apache-2.0 licenses. regulatory_restrictions: HIPAA compliance, 45 CFR 46 (Common Rule), institutional data privacy regulations, NIH Common Fund ethical AI requirements, export control regulations. No explicit confidentiality_level field in core schema.
quality Excellent governance documentation with clear access policy, licensing requirements, IP framework, and comprehensive regulatory restrictions. Core schema may lack explicit confidentiality_level field, but sensitivity clearly implied by controlled access requirement.
correctness regulatory_citations: HIPAA and 45 CFR 46 are correct US regulations for health data and human subjects; license_types: MIT and Apache-2.0 are valid open source licenses for software; access_mechanism: institutional email + DUA is standard controlled access pattern
consistency controlled_access_aligns_with_sensitivity: True; deidentified_but_controlled: correctly maintains controlled access despite de-identification due to re-id risk; ip_restrictions_match_license: NIH grant terms + institutional ownership consistent with controlled access; regulatory_requirements_match_data_type: HIPAA appropriate for clinical data
regulatory_restrictions

    regulatory_restrictions:
      id: chorus:regulatory:1
      name: HIPAA and human subjects research compliance
      description: 'Dataset subject to HIPAA (Health Insurance Portability and Accountability Act) compliance
        requirements for protected health information. Subject to 45 CFR 46 (Common Rule) for human subjects
        research protections. Institutional data privacy regulations at each of the 14 contributing sites
        apply. NIH Common Fund Bridge2AI program ethical and trustworthy AI requirements must be met. Export
        control regulations may apply for international users.

        '

✓ 1/1 R10 2.Dataset Access and Retrieval Regulatory Restrictions and Confidentiality Level Specified
evidence regulatory_restrictions: HIPAA compliance, 45 CFR 46 (Common Rule) for human subjects research, institutional data privacy regulations at 14 sites, NIH Bridge2AI ethical AI requirements, export control regulations for international users; confidential_elements: De-identified clinical data under controlled access model
quality Comprehensive regulatory framework with HIPAA, Common Rule, institutional requirements, and ethical AI standards; confidentiality level clearly specified as de-identified with controlled access
semantic Regulatory restrictions appropriate for multi-institutional clinical research dataset; export control mention suggests awareness of international data sharing constraints
5/5 R20 Q9 (Metadata Quality & Content) Access Requirements and Governance Documentation
level License + restrictions + confidentiality classification
evidence license_and_use_terms: controlled access requiring institutional email registration and signed licensing agreement, access request contacts provided. ip_restrictions: governed by CHoRUS licensing agreement and NIH grant terms (OT2OD032701), GitHub repos use MIT/Apache-2.0 licenses. regulatory_restrictions: HIPAA compliance, 45 CFR 46 (Common Rule), institutional data privacy regulations, NIH Common Fund ethical AI requirements, export control regulations. No explicit confidentiality_level field in core schema.
quality Excellent governance documentation with clear access policy, licensing requirements, IP framework, and comprehensive regulatory restrictions. Core schema may lack explicit confidentiality_level field, but sensitivity clearly implied by controlled access requirement.
correctness regulatory_citations: HIPAA and 45 CFR 46 are correct US regulations for health data and human subjects; license_types: MIT and Apache-2.0 are valid open source licenses for software; access_mechanism: institutional email + DUA is standard controlled access pattern
consistency controlled_access_aligns_with_sensitivity: True; deidentified_but_controlled: correctly maintains controlled access despite de-identification due to re-id risk; ip_restrictions_match_license: NIH grant terms + institutional ownership consistent with controlled access; regulatory_requirements_match_data_type: HIPAA appropriate for clinical data
is_deidentified

    is_deidentified:
      id: chorus:deid:1
      name: De-identified with controlled access
      description: 'All data de-identified before distribution. EHR data de-identified through OMOP CDM transformation
        and privacy scanning tools (privacy_scan_tool). Clinical notes tokenized via OHNLP toolkit with full
        text remaining local; only tokens available in enclave. Medical imaging undergoing DICOM header de-identification.
        Waveform and EEG data in WFDB/EDF+ formats with controlled access. Despite de-identification, controlled
        access model maintained due to sensitivity of clinical data and re-identification risk.

        '

⚠ low R10 · consistency
issue is_deidentified present with de-identification method described (OMOP transformation, OHNLP tokenization, DICOM header removal) but lacks explicit listing of removed identifiers
fields is_deidentified
fix Add explicit examples of identifiers removed (names, MRNs, dates, etc.)
✓ 1/1 R10 4.Ethical Use and Privacy Safeguards Deidentification Method Described
evidence is_deidentified: All data de-identified before distribution via OMOP CDM transformation and privacy_scan_tool for EHR, OHNLP tokenization for clinical notes (full text remains local, only tokens in enclave), DICOM header de-identification for imaging, WFDB/EDF+ formats with controlled access for waveforms/EEG
quality Comprehensive de-identification methods across all modalities: OMOP transformation + privacy_scan_tool (EHR), OHNLP tokenization (notes), DICOM header removal (imaging), controlled format conversion (waveforms/EEG); preprocessing_strategies detail these approaches
semantic De-identification methods semantically appropriate for data types: OMOP CDM transformation standardizes and de-identifies structured EHR, OHNLP tokenization removes identifiers from clinical text while preserving NLP utility, DICOM header cleaning removes embedded metadata
4/5 R20 Q8 (Metadata Quality & Content) Ethical and Privacy Declarations
level Very comprehensive ethics documentation
evidence ethical_reviews: IRB approvals at all 14 data acquisition centers, community-facing ethics focus groups. human_subject_research: retrospective critically ill patients (ICU, PICU, NICU), involves_human_subjects=true. informed_consent: waiver of consent framework with community ethics focus groups. is_deidentified: de-identified via OMOP transformation, privacy scanning tools, OHNLP tokenization. at_risk_populations: critically ill patients, pediatric (PICU), neonatal (NICU) vulnerable populations. No participant_compensation or participant_privacy fields in core schema.
quality Comprehensive ethics documentation covering IRB approvals, vulnerable populations, deidentification methods, and informed consent approach. Core schema may lack participant_compensation and explicit participant_privacy fields, but privacy protections described in is_deidentified.
correctness irb_plausibility: 14 acquisition centers requiring 14 IRB approvals is correct multi-site approach; deidentification_methods: OMOP transformation, privacy scanning, OHNLP tokenization appropriate for clinical data; vulnerable_population_identification: correctly identifies critically ill, pediatric, neonatal as vulnerable
consistency human_subjects_flag_matches_ethics: True; irb_count_matches_site_count: 14 acquisition centers = 14 IRB approvals (consistent); deidentification_aligns_with_data_types: different methods for EHR (OMOP), notes (OHNLP), imaging (DICOM header removal); consent_waiver_appropriate_for_retrospective: True
external_resources

    external_resources:
    - id: chorus:resource:1
      name: CHoRUS Project Website
      description: Official project website with dataset overview, team information, project components, and
        access instructions
    - id: chorus:resource:2
      name: CHoRUS GitHub Organization
      description: Comprehensive GitHub organization with 28 repositories including software, documentation,
        SOPs, and tooling at https://github.com/chorus-ai
    - id: chorus:resource:3
      name: Chorus_SOP Documentation Site
      description: Centralized standard operating protocol documentation with interactive workflow diagrams
        for data extraction and contribution at https://github.com/chorus-ai/Chorus_SOP
    - id: chorus:resource:4
      name: NIH RePORTER Project Details
      description: Federal grant information and project details from NIH Research Portfolio Online Reporting
        Tools for grant 1OT2OD032701-01 at https://reporter.nih.gov/project-details/10472824
    - id: chorus:resource:5
      name: Bridge2AI Program
      description: Parent NIH Common Fund program supporting AI-ready biomedical datasets across four data
        generation projects at https://bridge2ai.org/chorus
    - id: chorus:resource:6
      name: AIM-AHEAD Bridge2AI Training Program
      description: Partnership with AIM-AHEAD for Bridge2AI Clinical Care Training Program (Cohort 1 and Cohort
        2) providing AI/ML training for underrepresented trainees at https://aim-ahead.net/
    - id: chorus:resource:7
      name: OHDSI Community and OMOP CDM
      description: Observational Health Data Sciences and Informatics community supporting OMOP Common Data
        Model used for CHoRUS structured EHR data at https://www.ohdsi.org/
    - id: chorus:resource:8
      name: Published Research (Neurocritical Care)
      description: Peer-reviewed publication documenting CHoRUS dataset methodology and design in Neurocritical
        Care journal at https://doi.org/10.1007/s12028-024-02007

⚠ low R20 · completeness
issue Limited cross-platform data repository interlinking
fields external_resources
fix Add links to PhysioNet (WFDB format source), data commons platforms, or other repository registrations beyond project-specific infrastructure.
✓ 1/1 R10 1.Dataset Discovery and Identification Landing Page and Resources (page, hierarchical resources)
evidence page: https://chorus4ai.org/; external_resources with 8 entries including project website, GitHub organization (28 repos), SOP documentation, NIH RePORTER, Bridge2AI program, AIM-AHEAD training, OHDSI/OMOP community, peer-reviewed publication (doi: 10.1007/s12028-024-02007)
quality Active landing page plus comprehensive external resources with documentation, code repositories, training materials, and published literature
semantic Landing page and resources provide multiple access points for dataset understanding, technical documentation, and community engagement
✓ 1/1 R10 10.Cross-Platform and Community Integration Dataset Published on a Recognized Platform
evidence publisher: CHoRUS Consortium / Massachusetts General Hospital; distributions available at https://chorus4ai.org/ via secure enclave with controlled access; external_resources reference NIH RePORTER (https://reporter.nih.gov/project-details/10472824) and Bridge2AI program (https://bridge2ai.org/chorus)
quality Dataset published through CHoRUS Consortium infrastructure (secure enclave at chorus4ai.org) with NIH Bridge2AI program affiliation; not on traditional data repository platform (PhysioNet, Dataverse, Zenodo) but accessible through project-specific infrastructure with NIH program backing
semantic Publisher semantically appropriate: CHoRUS Consortium is multi-institutional collaboration led by MGH under NIH Bridge2AI program; controlled access via project-specific secure enclave reflects sensitive clinical data requirements
✗ 0/1 R10 10.Cross-Platform and Community Integration Citation and DOI for Cross-referencing
evidence No citation or doi fields in D4D-core schema; external_resources reference peer-reviewed publication (Neurocritical Care journal, doi: 10.1007/s12028-024-02007) documenting methodology but not dataset DOI; id field contains URI (https://chorus4ai.org/) but not DOI
quality D4D-core schema lacks citation and doi fields; publication DOI available for methodology paper (10.1007/s12028-024-02007) but no dataset DOI for direct citation; id field uses URI rather than persistent identifier
semantic Lack of dataset DOI limits cross-referencing and citation tracking; methodology paper DOI provides some citability but not direct dataset citation
✓ 1/1 R10 10.Cross-Platform and Community Integration Community Standards or Schema Conformance
evidence OMOP Common Data Model (OHDSI standard for observational health research), DICOM (medical imaging standard), WFDB (PhysioNet waveform standard), OHNLP (open source clinical NLP), EDF+ (European Data Format for EEG), Persyst (EEG format); external_resources link to OHDSI community (https://www.ohdsi.org/); preprocessing_strategies mention 'validated semantic mappings' in chorus-mapping repository with 'clinical validation SOP'
quality Comprehensive community standard conformance across all modalities: OMOP CDM (OHDSI ecosystem), DICOM (medical imaging interoperability), WFDB (PhysioNet physiological signal exchange), OHNLP (clinical NLP), EDF+ (neurophysiology); validated semantic mappings maintained in GitHub repository; external link to OHDSI community for tool ecosystem
semantic Standards conformance semantically robust: OMOP CDM enables cross-study observational research via OHDSI tools, DICOM enables imaging workflow integration, WFDB enables PhysioNet tool compatibility, OHNLP enables NLP research; validated mappings demonstrate quality assurance for standard conformance
✓ 1/1 R10 10.Cross-Platform and Community Integration Outreach Materials and Documentation Links
evidence external_resources with 8 entries: CHoRUS project website, GitHub organization (28 repos including software, documentation, SOPs, tooling at https://github.com/chorus-ai), Chorus_SOP documentation site with interactive workflow diagrams (https://github.com/chorus-ai/Chorus_SOP), NIH RePORTER project details, Bridge2AI program page, AIM-AHEAD training program (https://aim-ahead.net/), OHDSI community (https://www.ohdsi.org/), peer-reviewed publication (https://doi.org/10.1007/s12028-024-02007); existing_uses mention AIM-AHEAD training with Jupyter Notebooks and OHDSI/OMOP workshops
quality Extensive outreach and documentation: project website, GitHub organization with 28 repositories, standard operating procedure documentation with interactive diagrams, training program partnerships (AIM-AHEAD with Jupyter Notebooks and workshops), published methodology paper; documentation covers technical implementation (SOPs, GitHub repos), educational materials (training program, workshops), and scientific context (publication)
semantic Outreach materials semantically comprehensive: technical documentation (GitHub, SOPs) for developers, training materials (AIM-AHEAD, Jupyter Notebooks) for users, scientific publications for researchers, community links (OHDSI) for ecosystem integration
✗ 0/1 R10 10.Cross-Platform and Community Integration Related Datasets with Typed Relationships
evidence No related_datasets field in D4D-core schema; external_resources mention Bridge2AI program (https://bridge2ai.org/chorus) which includes 'four data generation projects' but no typed relationships to other Bridge2AI datasets
quality D4D-core schema lacks related_datasets field; CHoRUS is part of Bridge2AI program with other data generation projects (Voice, CM4AI, AI-READI) but relationships not formalized in metadata; full D4D schema includes related_datasets for typed relationships
semantic Related dataset relationships not documented in D4D-core; Bridge2AI program context suggests sibling relationships to other Bridge2AI datasets but lacks structured metadata
✓ 1/1 R10 2.Dataset Access and Retrieval Related Datasets and External Resources Linked
evidence external_resources with 8 entries: CHoRUS website, GitHub organization (28 repos), Chorus_SOP documentation, NIH RePORTER project details, Bridge2AI program, AIM-AHEAD training, OHDSI/OMOP community, peer-reviewed publication (Neurocritical Care journal); no related_datasets field in D4D-core
quality Extensive external resources linking to project infrastructure, community standards (OHDSI), training programs, and published research; D4D-core lacks related_datasets field
semantic External resources provide comprehensive context for dataset use; links to OHDSI community appropriate for OMOP CDM users
✓ 1/1 R10 8.Technical Transparency (Data Collection and Processing) External Standards and Resources Referenced
evidence external_resources with 8 entries: CHoRUS website, GitHub organization (28 repos), Chorus_SOP documentation, NIH RePORTER, Bridge2AI program, AIM-AHEAD training, OHDSI/OMOP community, peer-reviewed publication (doi: 10.1007/s12028-024-02007); standards mentioned throughout: OMOP CDM (OHDSI), DICOM, WFDB (PhysioNet), OHNLP, EDF+, Persyst, HIPAA, 45 CFR 46 (Common Rule)
quality Comprehensive external resources including community standards (OHDSI/OMOP), technical documentation (Chorus_SOP, GitHub repos), program context (Bridge2AI, AIM-AHEAD), regulatory frameworks (HIPAA, Common Rule), and published literature (Neurocritical Care journal); standards used throughout data lifecycle (collection, transformation, distribution)
semantic External standards semantically appropriate: OMOP CDM for observational health research, DICOM for medical imaging interoperability, WFDB for physiological signal exchange, OHNLP for clinical text NLP, EDF+ for neurophysiology; links to OHDSI community provide tool ecosystem access
3/5 R20 Q14 (Technical Documentation) Associated Publications
level One publication with DOI in external resources
evidence external_resources: Published Research (Neurocritical Care) at https://doi.org/10.1007/s12028-024-02007. Also lists CHoRUS website, GitHub organization (28 repos), NIH RePORTER, Bridge2AI program, AIM-AHEAD training, OHDSI community. No citation field in core schema.
quality One peer-reviewed publication with DOI listed in external resources. Multiple related resources documented. Core schema may lack dedicated citation field.
correctness doi_format: 10.1007/s12028-024-02007 matches valid DOI pattern (10.XXXX/...); doi_prefix: 10.1007 is Springer Nature registrar (valid for Neurocritical Care journal); external_urls: all URLs use valid HTTPS and plausible domains (github.com, nih.gov, bridge2ai.org)
consistency publication_aligns_with_project: Neurocritical Care journal appropriate for critical care dataset; external_resources_comprehensive: covers project website, code repos, funding, training programs; note: Core schema may lack citation field
1/1 R20 Q16 (FAIRness & Accessibility) Findability (Persistent Links)
level Pass - persistent URLs present
evidence page: https://chorus4ai.org/, external_resources: 8 resources with URLs including GitHub (https://github.com/chorus-ai), NIH RePORTER (https://reporter.nih.gov/project-details/10472824), Bridge2AI (https://bridge2ai.org/chorus), publication DOI (https://doi.org/10.1007/s12028-024-02007)
quality Multiple persistent URLs provided including project website, GitHub organization, federal grant database, and publication DOI. Strong findability through multiple access points.
correctness url_validity: all URLs use HTTPS protocol; domain_plausibility: chorus4ai.org, github.com, nih.gov, bridge2ai.org, doi.org are all valid domains; nih_reporter_id: 10472824 is plausible NIH project ID format
consistency page_matches_id: id and page both use https://chorus4ai.org/; external_urls_align_with_project: GitHub org chorus-ai matches project name CHoRUS
0/1 R20 Q20 (FAIRness & Accessibility) Interlinking Across Platforms
level Fail - limited cross-platform links
evidence external_resources: 8 resources including CHoRUS website, GitHub (28 repos at https://github.com/chorus-ai), NIH RePORTER, Bridge2AI, AIM-AHEAD, OHDSI community, publication. No links to PhysioNet, FAIRHUB, other data repositories, or data commons platforms.
quality Good linking to project infrastructure (GitHub, website) and parent programs (Bridge2AI, AIM-AHEAD) but lacking cross-platform data repository links. Not linked to PhysioNet despite using WFDB format, no data commons links, no other repository registrations. Score: Fail for limited cross-platform interlinking to data repositories.
correctness github_org: chorus-ai is valid GitHub organization name; external_urls: all provided URLs are valid HTTPS links; repository_count: 28 repositories in GitHub org is plausible
consistency external_resources_align_with_project: all listed resources are relevant to CHoRUS project; missing_expected_links: no PhysioNet link despite WFDB format use, no data repository registrations beyond project site

## Unmatched feedback

Feedback that referenced multiple fields or no specific field.
✗ 0/1 R10 1.Dataset Discovery and Identification Hierarchical Structure (parent datasets, relationships)
evidence No parent_datasets or related_datasets fields in D4D-core schema
quality D4D-core exchange layer subset does not include parent_datasets or related_datasets fields
semantic Expected limitation of D4D-core schema; full D4D schema includes these fields
✗ 0/1 R10 8.Technical Transparency (Data Collection and Processing) Software and Tools Documented
evidence Tools mentioned in text: OMOP CDM, OHDSI tool stack, OHNLP toolkit, privacy_scan_tool, DeGauss (UF-Geocoding tool), chorus-mapping repository, chorus_waveform repository, Chorus_SOP, CHoRUSReports; GitHub organization with 28 repositories; no software_and_tools field in D4D-core schema
quality D4D-core schema lacks software_and_tools field; tools mentioned throughout preprocessing_strategies, cleaning_strategies, external_resources (GitHub organization with 28 repos, Chorus_SOP documentation) but not in dedicated software metadata section
semantic Software/tool information present in narrative descriptions but not structured metadata; full D4D schema includes software_and_tools field for structured tool documentation
⚠ low R10 · schema_limitation
issue D4D-core subset lacks many fields expected by rubric10 sub-elements (doi, rrid, parent_datasets, related_datasets, download_url, format in top-level, conforms_to, variables, ethical_reviews detail, informed_consent detail, version, errata, was_derived_from, collection_mechanisms detail fields, software_and_tools, content_warnings, citation)
fields multiple
fix This is expected behavior for D4D-core exchange layer; full D4D schema contains these fields

## Recommendations

1. R10 · Add persistent identifier (DOI) from DataCite or similar registrar for dataset citation and cross-referencing; consider registering with PhysioNet or Zenodo for DOI assignment
2. R10 · Include explicit IRB institution names for all 14 acquisition centers in ethical_reviews to enhance ethical transparency
3. R10 · Add specific examples of identifiers removed during de-identification process (e.g., names, MRNs, dates per HIPAA Safe Harbor) to is_deidentified documentation
4. R10 · Consider adding formal version numbering scheme (e.g., v1.0 for Aug 2025 snapshot, v2.0 for 100K patient release) to support reproducible research and version tracking
5. R10 · Document conflicts of interest or financial disclosures for comprehensive ethical transparency
6. R10 · Add structured citation field with recommended citation format for dataset use in publications
7. R10 · Formalize relationships to other Bridge2AI datasets (Voice, CM4AI, AI-READI) using related_datasets field with typed relationships (e.g., 'is_part_of_program', 'shares_standards_with')
8. R10 · Consider adding software_and_tools field with version-pinned tool references (OMOP CDM version, OHNLP version, privacy_scan_tool version) to support reproducibility
9. R10 · Add errata or change log documentation for version-to-version differences as dataset expands from current 50K to target 100K+ admissions
10. R10 · Include provenance chain using was_derived_from to link to raw institutional EHR/PACS/monitor systems for full data lineage
11. R10 · Consider registering dataset with re3data.org or FAIRsharing.org to enhance discoverability in scientific data repository catalogs
12. R10 · Add RRID (Research Resource Identifier) for dataset citation in scientific publications (e.g., RRID:SCR_XXXXXX)
13. R10 · Document data use agreement text or link to full DUA for transparency about access terms beyond summary description
14. R10 · Add explicit grant number validation: confirm whether OT2OD032701 or 1OT2OD032701-01 is canonical award number for NIH RePORTER queries
15. R20 · Verify D4D-core schema structure and document which fields are schema limitations vs. content gaps (bytes, DOI, conforms_to, version, citation, etc.)
16. R20 · Add version numbers and GitHub repository URLs for all software tools: OHNLP toolkit version, OHDSI tools versions, privacy_scan_tool GitHub link, DeGauss version, chorus-mapping repo URL (https://github.com/chorus-ai/chorus-mapping), chorus_waveform repo URL (https://github.com/chorus-ai/chorus_waveform)
17. R20 · Enhance instances/subpopulations with demographic breakdowns to support diversity claims: age ranges, sex distribution, race/ethnicity, geographic distribution across 14 sites
18. R20 · If core schema supports it, add formal schema conformance references: conforms_to_schema with OMOP CDM version, DICOM standard version, WFDB/PhysioNet schema, OHNLP schema, EDF+ specification
19. R20 · Strengthen provenance tracking: add structured version history with timestamps, change logs documenting dataset expansions (e.g., v1.0 Sept 2022, v2.0 Aug 2025 with 50K admissions), errata documentation
20. R20 · Expand cross-platform interlinking: add PhysioNet link (https://physionet.org/ - WFDB format source), consider registering with data commons platforms, add links to OHDSI community resources beyond external_resources
21. R20 · If supported by schema, add explicit conforms_to field with formal schema URIs for OMOP CDM, DICOM, WFDB, OHNLP, EDF+ standards