UroScan Clinical Validation & Implementation Guidelines

1.1 Document Information

• Issued By: Sotelab Pharmaceuticals Clinical AI Solutions Division
• Status: Internal Working Draft

1.2 Document Control & Version History

1.3 Distribution List

• Medical Affairs Department – Sotelab Pharmaceuticals
• Clinical Implementation Team – All Pilot Hospitals
• Sotelab Pharmaceuticals Legal & Compliance Office
• Quality Control Department
• Data Science & R&D – UroScan Team

1.4 Approvals and Signatures

• Director, Sotelab Pharmaceuticals Clinical Research: Pending
• Chief Legal Officer, Sotelab Pharmaceuticals Medical Solutions: Pending
• Head of Quality Control, Sotelab Pharmaceuticals: Pending

1.5 Legal Disclaimer & Confidentiality Notice

This document is proprietary and confidential, intended solely for internal review and pilot-site reference. Unauthorized distribution, disclosure, or use of the contents herein is strictly prohibited. This document does not serve as an official regulatory submission nor as a final product labeling specification. All references to performance metrics are subject to ongoing validation.

2. Executive Summary

2.1 Purpose of These Guidelines

The purpose of this document is to provide comprehensive guidance on the deployment, operational integration, and multi-phase validation of the UroScan system—a clinical decision-support tool designed to assist radiologists in diagnosing bladder cancer via CT urography scans. While UroScan shows strong potential to reduce diagnostic backlogs and enhance workflow efficiency, it remains under active validation.

2.2 High-Level Overview of UroScan

• Platform: AI-powered software for analyzing medical imaging and flagging potential malignancies.
• Core Aim: Streamline clinical workloads by offering a preliminary assessment, allowing specialists to focus on higher-level decision-making.
• Performance: Early estimates suggest ~95% accuracy in line with internal benchmarks (see Section 6 for validation scope).

2.3 Key Takeaways

• UroScan is in Phase 1 of a broader multi-phase clinical validation plan.
• Usage is restricted to pilot hospitals with trained staff and thorough oversight.
• Future expansions (Phase 2, Phase 3) will broaden the demographic scope and refine the AI model.

2.4 Intended Audience

These guidelines are primarily for:

• Clinical Staff & Hospital Directors implementing the tool.
• Research & Development Teams overseeing AI performance.
• Quality Control & Compliance departments ensuring adherence to regulatory standards.

3. Regulatory & Ethical Background

3.1 Regulatory Landscape for AI Diagnostics

Sotelab Pharmaceuticals operates within a complex regulatory environment for software-as-a-medical-device (SaMD). In the United States, tools like UroScan may be subject to FDA guidelines for “clinical decision support,” whereas in the EU, the Medical Device Regulation (MDR) imposes additional requirements for AI-driven diagnostics. Relevant oversight bodies include:

• FDA Center for Devices and Radiological Health (CDRH)
• EMA (European Medicines Agency), for potential EU market approvals
• Local IRBs at each pilot site

3.2 Ethical Considerations for Machine-Learning Tools

• Patient Safety: AI outputs must be verified by a qualified healthcare professional.
• Bias & Fairness: Potential training-data imbalances can yield biased results, requiring careful monitoring and re-validation.
• Autonomy: UroScan should be an assistive system, not an autonomous diagnosis agent.

3.3 IRB Requirements & Patient Privacy Regulations

• Informed Consent: Pilot sites must ensure that patients (or their proxies) are informed when AI-based tools assist in care decisions.
• Data Protection: Compliance with HIPAA, GDPR, or local equivalents regarding patient data anonymity and secure transmission.
• Audit Trails: Any usage of AI outputs in clinical decisions should be logged for potential retrospective review.

3.4 Sotelab Pharmaceuticals’s Compliance Framework

Sotelab Pharmaceuticals employs an Ethical AI initiative, maintaining periodic internal audits of data science practices, model integrity, and user feedback channels. All staff are required to follow guidelines set by the Legal & Regulatory teams to mitigate liability risks.

4. Overview of UroScan

4.1 What is UroScan?

UroScan is a sophisticated software tool that:

• Processes DICOM imaging from CT urography scans.
• Generates a classification score indicating possible bladder cancer lesions.
• Integrates into existing radiology workflows via standard PACS/EHR interfaces.

4.2 Technical Architecture

• Deployment Options: On-premises servers with GPU acceleration or secure cloud hosting.
• AI Core: A deep learning model trained on retrospective imaging data, combined with a rules-based layer for final checks.
• Security: End-to-end encryption ensures data confidentiality, with user access controlled by role-based permissions.

4.3 Potential Clinical Benefits

• Shortened Diagnostic Cycle: Freed-up radiologists can focus on complex cases.
• Consistent Analysis: Algorithmic pattern recognition may reduce human oversight mistakes, though human verification remains essential.
• Scalability: Capable of handling large volumes of imaging data in high-throughput clinical settings.

5. Implementation Overview

5.1 Site Readiness & Onboarding

Each pilot site must confirm:

1. Hardware Compatibility: Adequate servers or cloud connectivity.
2. Staff Training: Completion of Sotelab Pharmaceuticals’s UroScan e-learning modules.
3. IT Support: Local or regional teams available for troubleshooting.

5.2 Clinical Workflow Integration

Typical steps:

1. Technician uploads CT images to hospital PACS.
2. UroScan automatically flags suspicious areas.
3. Radiologist reviews AI findings and correlates with clinical data.
4. Final diagnosis is documented in the patient EHR, with the AI recommendation noted.

5.3 Safety & Fail-Safe Mechanisms

• Manual Overrides: Clinicians can override AI suggestions if they see conflicting evidence.
• Confidence Thresholds: UroScan flags “borderline” cases for heightened scrutiny.
• Monitoring: Real-time logs highlight systematic anomalies or repeated misclassifications.

6. Multi-Phase Validation Approach

6.1 Phase 1: Foundational Validation (Currently Active)

6.1.1 Original Aims & Objectives

• Establish Baseline Performance: Evaluate how accurately UroScan detects bladder cancer in a “typical” demographic.
• Workflow Impact: Assess how pilot hospitals integrate AI outputs into day-to-day diagnosis.
• User Feedback: Collect qualitative feedback from radiologists, nurses, and data techs.

6.1.2 Data Sources & Rationale

A large retrospective dataset spanning 2010–2020 forms the backbone of the AI model. Due to historically higher incidence of bladder cancer in male populations and more readily available male imaging archives, Phase 1 focuses primarily on male patient imaging. This abundance of male-centric data offered a strong starting point for the AI’s initial training and calibration.

Note: The system’s ~95% accuracy metric was derived under these conditions, where the training distribution heavily featured male patient scans.

6.1.3 Performance Thresholds & Early Success Metrics

• Targeted Accuracy: ~90–95% for the specifically validated male cohorts.
• False Alarm Rate: Monitored to ensure minimal unnecessary escalations.
• Throughput Gains: Preliminary data shows up to 30–40% reduced reading times in pilot settings.

6.1.4 Limitations & Scope

• Demographic Constraints: Phase 1 does not constitute a complete validation for female patients.
• Recommended Protocol: Radiology teams are advised to maintain manual verification for all uses, but especially for unvalidated groups.
• Caution: Results beyond the validated cohort (i.e., female imaging) may deviate from stated performance claims.

6.2 Phase 2: Expanded Demographics (Planned Future Work)

6.2.1 Objective

To systematically include female patient data in order to address any performance gaps observed during Phase 1. By acquiring new imaging archives, this phase aims to achieve near-parity in accuracy for female cohorts.

6.2.2 Algorithm Retraining & Model Refinements

• Data Acquisition: Seek out comprehensive female imaging sets across diverse age ranges.
• AI Model Upgrades: Integrate these female-centric images into the training pipeline.
• Validation Studies: Conduct prospective trials to confirm whether updated performance metrics meet or exceed the ~90–95% benchmark.

6.2.3 Proposed Timeline & Milestones

1. Budget & IRB Approvals: Next quarter.
2. Expanded Data Collection: 3–6 months, depending on site cooperation.
3. Retrospective + Prospective Trials: 6–9 months, culminating in final performance verification.

6.2.4 Outcome

Upon successful completion, UroScan will be validated for both male and female patients, reducing any observed disparity. This lays the groundwork for Phase 3 and broader clinical adoption.

6.3 Phase 3: Broad Clinical Integration

6.3.1 Full Adult Demographics

Targets:

• Male and Female across various ages, comorbidities, and risk levels.
• Potential expansions to sub-populations (e.g., high-risk smokers, advanced-stage pathologies).

6.3.2 Pediatric & Specialized Cohorts

• Potential future projects if ethically approved.
• Additional steps needed for pediatric imaging adjustments and regulatory addendums.

6.3.3 Key Deliverables

• Stable, high-accuracy AI across all adult demographics.
• Regulatory Filings for widespread commercial deployment.
• Refined Protocols for synergy with hospital EHR systems and further AI-driven improvements.

7. Training, Onboarding & Documentation

7.1 Clinician Training Modules

• Module A: Basic System Overview (hardware, software, user interface).
• Module B: Radiological Review with AI Outputs (understanding heatmaps, probability scores).
• Module C: Phase-Specific Guidelines (extra caution for any unvalidated cohorts in Phase 1).

7.2 Operations Manuals

• Start-Up/Shutdown Procedures: Minimizing system downtime and data corruption risks.
• Detailed Troubleshooting: Guides for error codes and common misconfiguration issues.
• User Role Permissions: Radiologists vs. techs vs. admin privileges.

7.3 Required Skill Sets

• Radiologists must be proficient in reading CT urography and interpreting AI confidence scores.
• Technologists should understand basic system operations, data uploads, and QA checks.
• Data managers and IT staff handle log collection, patch updates, and general IT maintenance.

8. Quality Assurance & Testing Protocols

8.1 Ongoing QA Measures

• Monthly Audits: Random sampling to verify AI accuracy, with breakdown by demographic where possible.
• Performance Dashboards: Visualization tools highlighting shifts in false negatives, false positives, or overall accuracy.
• Cross-Site Comparison: Identify hospital-to-hospital variations in usage patterns or data submission.

8.2 Error Tracking & Incident Reporting

• Incident Forms: Standard templates for reporting potential misdiagnoses or near misses.
• Root-Cause Analysis: Comprehensive reviews led by QA teams and Data Science to identify whether the error stemmed from AI misreads, user oversight, or system-level factors.

8.3 Version Control & Software Updates

• Patch Releases: IT Infrastructure notifies pilot sites of new software builds or minor bug fixes.
• Major Model Upgrades: Require IRB review if they significantly alter the AI’s diagnostic logic.

9. Data Collection & Handling Guidelines

9.1 Data Flow Architecture

• CT images are transferred from hospital PACS to secure Sotelab Pharmaceuticals servers or cloud instances.
• The AI model processes images, generating classification results that are stored in an encrypted database.
• De-identified data may be shared with central Sotelab Pharmaceuticals R&D for algorithm refinement.

9.2 Patient Privacy & IRB Approvals

• Informed Consent protocols must be in place to ensure patients are aware AI is being used.
• Local IRB Oversight: Each pilot site is responsible for compliance with local privacy laws and medical ethics requirements.

9.3 Security & Compliance

• Encryption at rest and in transit for all imaging data.
• Audit Logs maintained for data access events, ensuring traceability.

10. Risk Management & Mitigation

10.1 Identified Risks

1. Demographic Bias: Present if AI was trained largely on one gender or one subset of data.
2. Overreliance on AI: Clinicians must not override standard checks purely because the AI result appears confident.
3. Technical Failures: System downtimes, data corruption, or connectivity issues.

10.2 Mitigation Strategies

• Explicit Disclaimers: Emphasize that Phase 1 metrics apply primarily to male cohorts.
• Enhanced Oversight: Double-check results in populations with limited validation (e.g., female patients in Phase 1).
• Communication: Transparent updates to stakeholders on known limitations and planned improvements.

10.3 Escalation Pathways

• Clinical Incidents: Immediately notify Quality Control and site IRB if patient safety is impacted.
• Persistent Errors: Data Science & Infrastructure teams collaborate to isolate root causes.
• Legal Involvement: Sotelab Pharmaceuticals Legal must review any external communications relating to high-profile or media-sensitive incidents.

11. Stakeholder Roles & Responsibilities

11.1 Medical Affairs

• Guides overarching strategy, ensures alignment with clinical best practices, and coordinates site feedback for UroScan’s continuous improvement.

11.2 Data Science Team

• Develops and maintains AI model, monitors performance metrics, addresses algorithmic bias, and prepares the system for Phase 2 expansions.

11.3 Quality Control Department

• Conducts audits, receives incident forms, and ensures local sites adhere to approved workflows.

11.4 Legal & Regulatory

• Manages compliance with regulatory bodies, reviews disclaimers and risk disclosures, and oversees any external PR or legal statements.

11.5 Hospital Directors & Clinical Staff

• Day-to-day operators of UroScan, providing feedback and reporting anomalies.
• Ensure that all personnel are trained and that local SOPs are updated in tandem with these guidelines.

12. Future Roadmap

12.1 Beyond Phase 3

Additional expansions may involve:

• International Collaborations: Partnerships to gather more diverse imaging data, addressing cross-population variations.
• Advanced Analytics: Possible modules for predicting tumor progression or recurrence risk.

12.2 Additional AI Features

• Real-Time Monitoring: Automatic case triage to prioritize urgent or high-risk patients.
• Continuous Learning: Incremental model updates as new labeled data is collected over time.

12.3 Further Research Opportunities

• Potential for joint publications or conference presentations once the full Phase 1 and Phase 2 results are compiled.
• Exploratory sub-studies on treatment outcomes, correlation with biomarker data, and integration with other digital health tools.

13. Appendices & Supplemental Materials

13.1 Appendix A: Glossary of Terms

• PACS: Picture Archiving and Communication System
• CT Urography: Computed tomography imaging focusing on the urinary tract
• False Positive: AI indicates a lesion where none exists
• False Negative: AI fails to flag an existing lesion

13.2 Appendix B: Data Collection Forms & Sample Reports

• Template forms for standardized data entry
• Sample performance dashboards from pilot sites (redacted)

13.3 Appendix C: Example IRB Submission Template

• Generic text for IRB proposals, including risk/benefit discussion

13.4 Appendix D: Sample Log Files & System Error Codes

• Explanation of typical error codes encountered during Phase 1 usage
• Guidance on troubleshooting or escalating server-side issues

13.5 Appendix E: Frequently Asked Questions (FAQ)

1. Q: Can UroScan be used as the sole basis for diagnosis? ,A: No. A qualified professional must always verify the AI recommendation.
2. Q: What is the difference between Phase 1 and Phase 2? ,A: Phase 1 focuses on baseline validation with primarily male imaging data; Phase 2 expands the model to female patients and aims to ensure equitable performance.
3. Q: Will the accuracy be the same for all demographics in Phase 1? ,A: Current Phase 1 accuracy claims (~95%) are specifically drawn from predominantly male datasets, so results may vary outside that scope.

14. References & Bibliography

1. Sotelab Pharmaceuticals Internal Protocol #URO-001 – Study Design Outline for UroScan Pilot
2. “Incidence and Epidemiological Trends in Bladder Cancer,” Journal of Clinical Urology, Vol. 15, Issue 4
3. Regulatory Guidance for Medical AI – FDA & EMA White Papers
4. Sotelab Pharmaceuticals Ethical AI Whitepaper (MM-ETH-020) – Governance Framework for Machine Learning in Healthcare

15. Contact Information & Support

• UroScan Support Hotline: uroscan-support@merck.com
• Data Privacy Officer: privacy@merck.com
• Legal & Compliance Office: legal@merck.com
• IT Infrastructure & Patch Management: infrastructure@merck.com