Rubric10-Semantic Evaluation Report

Dataset

Unknown

Generation Method

Unknown

Evaluator

Unknown

Rubric Type

Unknown

Temperature

N/A

Evaluation Date

Unknown

0/50

0.0% Overall Score

Element Scores

Element None: None

0/5 (0%)

Persistent Identifier (DOI, RRID, or URI) 1/1

Fields:

Rationale: id: https://doi.org/10.13026/37yb-1t42 (line 7)

Dataset Title and Description Completeness 1/1

Fields:

Rationale: title: Bridge2AI-Voice - An ethically-sourced, diverse voice dataset linked to health information (line 9); description: comprehensive 17-line description (lines 10-26)

Keywords or Tags for Searchability 1/1

Fields:

Rationale: keywords: 45 keyword entries covering voice biomarker, disorders, methods, standards (lines 29-72)

Landing Page and Resources (page, hierarchical resources) 1/1

Fields:

Rationale: page: https://docs.b2ai-voice.org (line 27); external_resources: 11 resource entries (lines 708-765)

Hierarchical Structure (parent datasets, relationships) 1/1

Fields:

Rationale: External resources section implicitly documents hierarchical relationships through platform linkages (PhysioNet, Health Data Nexus, Zenodo)

Element None: None

0/5 (0%)

Access Policy and IP Restrictions Defined 1/1

Fields:

Rationale: license_and_use_terms: comprehensive section with license terms list (lines 508-531)

Regulatory Restrictions and Confidentiality Level Specified 1/1

Fields:

Rationale: Certificate of Confidentiality protections documented (lines 528, 696-705); HIPAA Safe Harbor de-identification (lines 418-430)

Download URL or Platform Link Available 1/1

Fields:

Rationale: access_urls in distribution_formats: https://physionet.org/content/b2ai-voice/ (lines 541, 549, 557, 565)

Distribution Formats and File Types Specified 1/1

Fields:

Rationale: distribution_formats: 5 format entries covering Parquet (spectrograms, MFCCs), TSV (phenotype, static features), and raw audio (lines 533-573)

Related Datasets and External Resources Linked 1/1

Fields:

Rationale: external_resources: 11 entries covering PhysioNet, GitHub, NIH RePORTER, Health Data Nexus, Zenodo, publications, DACO contact (lines 708-765)

Element None: None

0/5 (0%)

License Terms Allow Reuse 1/1

Fields:

Rationale: license_and_use_terms with explicit reuse provisions: public disclosure encouraged, recognition required, open-access publication preferred (lines 526-527)

Data Formats Are Standardized (encoding, format) 1/1

Fields:

Rationale: Standardized formats: Parquet for arrays (spectrograms, MFCCs), TSV for tabular data, JSON for dictionaries (lines 533-573)

Schema or Ontology Conformance Stated 1/1

Fields:

Rationale: Schema conformance documented in header: Schema: src/data_sheets_schema/schema/data_sheets_schema_all.yaml (line 4); FAIR principles mentioned in keywords (line 70)

Variable Metadata with Identifiers Defined 1/1

Fields:

Rationale: Data dictionaries provided: phenotype.json for demographics/questionnaires, static_features.json for acoustic features (lines 555, 563)

Use Guidance Provided (intended, prohibited uses) 1/1

Fields:

Rationale: intended_uses: 5 use cases (lines 454-480); discouraged_uses: 4 discouraged use cases (lines 482-504)

Element None: None

0/5 (0%)

IRB or Ethics Review Documented 1/1

Fields:

Rationale: human_subject_research: comprehensive section documenting USF IRB approval, informed consent, bioethics guidance (lines 641-658)

Deidentification Method Described 1/1

Fields:

Rationale: HIPAA Safe Harbor de-identification comprehensively documented with 18 identifier categories removed (lines 418-430)

Privacy Protections Beyond Deidentification 1/1

Fields:

Rationale: Certificate of Confidentiality (lines 696-705); raw audio controlled access (lines 239-245, 664-671); free speech transcript removal (lines 399-405)

Informed Consent Obtained from Participants 1/1

Fields:

Rationale: Written informed consent documented for data collection initiative, data sharing, EHR access, and research data sharing (lines 645-648)

Vulnerable Populations and Compensation Documented 1/1

Fields:

Rationale: Pediatric cohort with special ethical considerations documented (lines 296-301); additional privacy precautions for pediatric data noted (line 608)

Element None: None

0/5 (0%)

Cohort or Subpopulations Characteristics Described 1/1

Fields:

Rationale: subpopulations: 5 cohorts detailed (Voice Disorders, Neurological, Mood/Psychiatric, Respiratory, Pediatric) with clinical characteristics (lines 270-301)

Number of Instances or Samples Reported 1/1

Fields:

Rationale: instances: 306 participants, 12,523 recordings documented (lines 214-225)

Variable-Level Metadata and Tabular Flag 1/1

Fields:

Rationale: JSON data dictionaries for phenotype and static features (lines 555, 563); tabular data explicitly documented as TSV format

Data Topics or Conditions Represented 1/1

Fields:

Rationale: Five disease cohort categories detailed in instances and subpopulations sections (lines 220-225, 270-301)

Data Quality Issues and Anomalies Documented 1/1

Fields:

Rationale: sampling_strategies: detailed sampling methodology with non-random, non-representative sampling explicitly stated (lines 247-268)

Element None: None

0/5 (0%)

Dataset Version Number Provided 1/1

Fields:

Rationale: Version progression documented: v1.0, v1.1, v2.0.0, v2.0.1 with dates (lines 603-615)

Version Access Methods Documented 1/1

Fields:

Rationale: Version-specific documentation and DOI system mentioned (lines 606, 619); PhysioNet platform provides versioned access

Change Descriptions and Errata Provided 1/1

Fields:

Rationale: updates: detailed version history with specific changes (v1.1 added MFCCs, v2.0.0 expanded cohort, etc.) lines 599-619

Update Schedule or Frequency Indicated 1/1

Fields:

Rationale: frequency: Periodic versioned releases during data collection period (2022-2026) (line 609); future releases planned (lines 616-617)

Provenance and Source Derivation Documented 1/1

Fields:

Rationale: Generation provenance in header: Source: data/preprocessed/concatenated/VOICE_preprocessed.txt (89K, 9 source files) (line 3); preprocessing strategies document feature derivation (lines 345-415)

Element None: None

0/5 (0%)

Motivation or Purpose for Dataset Creation 1/1

Fields:

Rationale: purposes: 3 comprehensive purpose statements covering clinical integration, flagship dataset creation, and standards development (lines 76-93)

Primary Research Objectives or Tasks 1/1

Fields:

Rationale: tasks: 6 task entries covering AI/ML development for voice disorders, neurological, mood, respiratory, pediatric disorders, and workforce development (lines 95-128)

Funding Sources and Mechanisms Listed 1/1

Fields:

Rationale: funders: 2 entries - NIH Office of the Director and NIBIB with detailed grant information (lines 195-210)

Grant IDs or Award Numbers Present 1/1

Fields:

Rationale: Grant numbers: 3OT2OD032720-01S3 (NIH OD), R01EB030362 (NIBIB) with opportunity number OTA-21-008 (lines 199, 208)

Creators and Acknowledgements Documented 1/1

Fields:

Rationale: creators: 13 entries including contact PI, co-PIs, co-investigators, and consortium (lines 154-193)

Element None: None

0/5 (0%)

Collection Mechanisms and Settings Described 1/1

Fields:

Rationale: collection_mechanisms: 3 detailed entries covering smartphone app protocol, multi-institutional collection, and comprehensive data collection protocol (lines 304-323)

Data Acquisition Methods Listed 1/1

Fields:

Rationale: acquisition_methods: 3 entries covering voice recordings, questionnaires, and EHR access (lines 325-342)

Preprocessing, Cleaning, and Labeling Strategies 1/1

Fields:

Rationale: preprocessing_strategies: 7 entries with detailed technical specifications (lines 345-415); cleaning_strategies: 3 entries (lines 417-451)

Software and Tools Documented 1/1

Fields:

Rationale: Software tools documented throughout preprocessing: OpenSMILE, Praat, Parselmouth, Whisper, b2aiprep library, REDCap (lines 378-415)

External Standards and Resources Referenced 1/1

Fields:

Rationale: external_resources: 11 entries including standards publications, platforms, repositories (lines 708-765); FAIR principles in keywords (line 70)

Element None: None

0/5 (0%)

Known Limitations Documented 1/1

Fields:

Rationale: Limitations implicitly documented: non-representative sampling (line 263), pediatric data not yet released (line 301), multi-session requirements for subset of participants (lines 308-310)

Systematic Biases Identified and Described 0/1

Fields:

Rationale: No explicit known_biases field present

Data Anomalies and Quality Issues Noted 0/1

Fields:

Rationale: No explicit anomalies field present

Sensitive Content and Warnings Provided 1/1

Fields:

Rationale: sensitive_elements: 4 comprehensive entries covering voice biometrics, EHR data, demographic data, and Certificate of Confidentiality (lines 660-705)

Ethical Review Details Including Conflicts 1/1

Fields:

Rationale: human_subject_research includes IRB approval and ethics review board identification (lines 655-658); bioethics guidance integration documented (lines 649-653)

Element None: None

0/5 (0%)

Dataset Published on a Recognized Platform 1/1

Fields:

Rationale: PhysioNet distribution documented throughout (lines 207-210, 533-573, 592-596, 710-713); Health Data Nexus alternative platform (lines 730-733)

Citation and DOI for Cross-referencing 1/1

Fields:

Rationale: DOI: https://doi.org/10.13026/37yb-1t42 (line 7); recognition of data source required in publications (line 527)

Community Standards or Schema Conformance 1/1

Fields:

Rationale: FAIR principles (line 70), CARE principles (line 71), D4D schema conformance (line 4), PhysioNet requirements (line 518)

Outreach Materials and Documentation Links 1/1

Fields:

Rationale: page: https://docs.b2ai-voice.org (line 27); external_resources: comprehensive documentation including GitHub, publications, NIH RePORTER (lines 708-765)

Related Datasets with Typed Relationships 0/1

Fields:

Rationale: No explicit related_datasets field with typed relationships present

Semantic Analysis

Strengths

Exceptional completeness across all discovery, access, and retrieval dimensions (100% for elements 1-2)
Outstanding ethical documentation including IRB approval, informed consent, deidentification, and privacy protections beyond standard requirements
Comprehensive technical transparency with detailed preprocessing strategies documenting 7 processing steps with technical parameters
Excellent funding and motivation transparency with complete grant information, creator attribution, and scientific objectives
Strong data composition documentation with 5 subpopulations, instance counts, and sampling methodology
Robust version tracking with 4 documented releases and clear update schedule through 2026
Extensive external resources section with 11 documented links to platforms, documentation, and publications
Well-documented use guidance with 5 intended uses and 4 discouraged uses covering ethical considerations
Comprehensive license terms with 12 specific provisions for registered access and data use

Weaknesses

No explicit known_biases field to document systematic biases, though non-representative sampling is acknowledged
No explicit anomalies field to document data quality issues or outliers
No explicit related_datasets field with typed relationships to other datasets in the Bridge2AI program or broader ecosystem
While limitations are documented throughout, they are not consolidated in a dedicated known_limitations field for easy reference

Recommendations

Add explicit known_biases section to document potential biases from non-representative sampling, specialty clinic recruitment, and cohort selection

Create anomalies section to document any data quality issues encountered during collection or processing

Add related_datasets field with typed relationships to other Bridge2AI datasets (AI-READI, CHORUS, CM4AI) and complementary voice datasets

Consolidate limitations into dedicated known_limitations field while retaining context-specific documentation

Consider adding explicit publisher field naming PhysioNet/MIT as primary publisher organization

Generated on 2025-12-20 19:23:41 using Bridge2AI Data Sheets Schema