| Title: | AI Agents, MCP Tooling, and Clinical NLP for 'DICOM' Imaging |
|---|---|
| Description: | Satellite Artificial Intelligence ('AI') and clinical natural language processing toolkit for 'dicomtk'. Provides 'ellmer'-based clinical imaging agents with unbypassable monotonic de-identification guardrails, a Model Context Protocol ('MCP') server via 'mcptools', a pluggable REST and offline backend registry for segmentation models ('TotalSegmentator' by Wasserthal et al. (2023) <doi:10.1148/ryai.230024>, 'nnU-Net', 'VISTA3D', and 'MedSAM2' by Ma et al. (2024) <doi:10.1038/s41467-024-44824-z>), Integrating the Healthcare Enterprise AI Results ('IHE AIR') standardized 'DICOM SEG' and 'TID 1500' outputs, radiology report de-identification and weak-label extraction, 'RadLex' and 'LOINC-RSNA' protocol harmonisation, and 'DuckDB' vector similarity search. |
| Authors: | Yee How Lau [aut, cre, cph] |
| Maintainer: | Yee How Lau <[email protected]> |
| License: | GPL (>= 2) |
| Version: | 0.2.0 |
| Built: | 2026-11-14 15:54:00 UTC |
| Source: | https://github.com/lauyeehow1986-hub/dicomtk.ai |
Stage 3 of the agent loop. Requires explicit approval before any mutation or de-identification output is written.
ai_agent_approve(dry_run, auto_approve = FALSE, approver = NULL)ai_agent_approve(dry_run, auto_approve = FALSE, approver = NULL)
dry_run |
A |
auto_approve |
Logical. Set |
approver |
Optional character name/identifier of the approving user. |
Updated "dicom_agent_plan" object at stage = "approved".
Stage 2 of the agent loop. Evaluates effective tag actions and optional image QC without writing or mutating any files on disk.
ai_agent_dry_run(plan)ai_agent_dry_run(plan)
plan |
A |
Updated "dicom_agent_plan" object at stage = "dry_run".
Stage 4 of the agent loop. Re-verifies the non-weakening guardrail and
executes de-identification and/or QC into out_dir.
ai_agent_execute(approved_plan, out_dir)ai_agent_execute(approved_plan, out_dir)
approved_plan |
An approved |
out_dir |
Output directory for de-identified files and provenance logs. |
Updated "dicom_agent_plan" object at stage = "executed".
Stage 1 of the 5-stage clinical agent loop (plan -> dry-run -> approve -> execute -> verify). Enforces the non-weakening de-identification guardrail
up front.
ai_agent_plan( task = c("deid_and_qc", "deid_only", "qc_only"), path, baseline_profile = dicomtk::dicom_deid_profile(), override_tags = list(), options = character(0), chat = ai_chat("offline") )ai_agent_plan( task = c("deid_and_qc", "deid_only", "qc_only"), path, baseline_profile = dicomtk::dicom_deid_profile(), override_tags = list(), options = character(0), chat = ai_chat("offline") )
task |
Workflow task: |
path |
Input DICOM file or directory path. |
baseline_profile |
Baseline |
override_tags |
Optional named list of tag-action overrides (must not
weaken |
options |
Optional character vector of PS3.15 options. |
chat |
A |
An S3 "dicom_agent_plan" object at stage = "planned".
synth <- dicomtk::dicom_synthesize(modality = "CT", dims = c(8L, 8L, 2L)) on.exit(unlink(synth$out_dir, recursive = TRUE), add = TRUE) plan <- ai_agent_plan("deid_only", path = synth$out_dir) dry <- ai_agent_dry_run(plan)synth <- dicomtk::dicom_synthesize(modality = "CT", dims = c(8L, 8L, 2L)) on.exit(unlink(synth$out_dir, recursive = TRUE), add = TRUE) plan <- ai_agent_plan("deid_only", path = synth$out_dir) dry <- ai_agent_dry_run(plan)
Stage 5 of the agent loop. Runs dicomtk::dicom_deid_verify() on the
executed output directory to confirm zero residual PHI leakage.
ai_agent_verify(exec_result)ai_agent_verify(exec_result)
exec_result |
A |
Updated "dicom_agent_plan" object at stage = "verified" with
$passed logical status.
Probes a registered backend's readiness ("offline_ready" for offline
backends, or HTTP GET /health / base URL for REST backends).
ai_backend_health(name, timeout_sec = 3)ai_backend_health(name, timeout_sec = 3)
name |
Backend name. |
timeout_sec |
HTTP timeout in seconds for REST backends (default |
A list with name, mode, status, and endpoint.
ellmer-Backed Chat Session for dicomtk.ai
Creates a "dicom_ai_chat" object wrapping an ellmer chat client (or a
deterministic "offline" rule-based planner for CI and air-gapped testing).
ai_chat( provider = c("offline", "ollama", "openai", "anthropic", "gemini", "openai_compatible"), model = NULL, base_url = NULL, api_key = NULL, system_prompt = NULL, ... )ai_chat( provider = c("offline", "ollama", "openai", "anthropic", "gemini", "openai_compatible"), model = NULL, base_url = NULL, api_key = NULL, system_prompt = NULL, ... )
provider |
Chat provider: |
model |
Optional model identifier. |
base_url |
Optional API base URL (used for |
api_key |
Optional API key (defaults to standard provider environment variables). |
system_prompt |
Optional system prompt string. |
... |
Additional arguments passed to the underlying |
An object of class "dicom_ai_chat".
chat <- ai_chat("offline")chat <- ai_chat("offline")
Fit and Apply Empirical Bayes ComBat Harmonisation
ai_combat_harmonize(features, batch, mod = NULL, eb = TRUE, ref_batch = NULL)ai_combat_harmonize(features, batch, mod = NULL, eb = TRUE, ref_batch = NULL)
features |
Numeric matrix ( |
batch |
Factor or character vector of scanner/site batch labels of
length |
mod |
Optional biological covariate design matrix ( |
eb |
Logical; if |
ref_batch |
Optional reference batch name. |
A list with harmonized (harmonized matrix) and fit
("dicom_combat_fit").
set.seed(1) mat <- matrix(rnorm(40), nrow = 4, ncol = 10) b <- rep(c("A", "B"), each = 5) res <- ai_combat_harmonize(mat, b) dim(res$harmonized)set.seed(1) mat <- matrix(rnorm(40), nrow = 4, ncol = 10) b <- rep(c("A", "B"), each = 5) res <- ai_combat_harmonize(mat, b) dim(res$harmonized)
Redacts or pseudonymizes protected health information (PHI) in free-text
radiology reports or dicom_sr objects using a registered NLP backend
("presidio" REST or built-in deterministic HIPAA/GDPR/SG pattern NER).
Enforces a monotonic union invariant: optional LLM audit passes can only add
missed spans, never remove deterministic detections.
ai_deid_report( x, backend = "presidio", chat = NULL, mode = c("redact", "pseudonymize") )ai_deid_report( x, backend = "presidio", chat = NULL, mode = c("redact", "pseudonymize") )
x |
Character report text or a |
backend |
Registered NLP de-id backend name (default |
chat |
Optional |
mode |
|
A list with text (de-identified string), spans (tibble of
detected PHI spans), and mode.
ai_deid_report("Patient John Tan (MRN: MRN-12345) scanned on 2026-03-14.")ai_deid_report("Patient John Tan (MRN: MRN-12345) scanned on 2026-03-14.")
Computes an L2-normalized feature embedding from a dicom_volume using a
registered embedding backend ("offline" deterministic intensity/spatial/modality
signature, or a REST foundation-model endpoint such as RAD-DINO / BiomedCLIP).
ai_embed(vol, backend = "embeddings", dim_out = 32L)ai_embed(vol, backend = "embeddings", dim_out = 32L)
vol |
A |
backend |
Registered embedding backend name (default |
dim_out |
Target embedding dimension for the offline extractor
(default |
A numeric L2-normalized embedding vector.
Open or Initialize a DuckDB Study Embedding Index
ai_embedding_index(db_path = ":memory:")ai_embedding_index(db_path = ":memory:")
db_path |
Path to a DuckDB database file (or |
A "dicom_ai_index" object.
Writes segmentation masks as a DICOM SEG object (write_dicom_seg()) and
quantitative summary metrics as a DICOM TID 1500 Measurement Report
(dicom_sr_tid1500()), aligned with IHE AI Results (AIR) conventions.
ai_export_results( seg_res, vol, out_dir, title = "IHE AIR AI Segmentation Report" )ai_export_results( seg_res, vol, out_dir, title = "IHE AIR AI Segmentation Report" )
seg_res |
A |
vol |
Reference |
out_dir |
Output directory for standardized DICOM files. |
title |
Report title for the TID 1500 Measurement Report. |
Invisibly returns a list with seg_path, sr_path, out_dir, and
metrics.
Extracts structured pathology findings with assertion polarity ("present",
"absent", or "uncertain") using deterministic ConText negation/uncertainty
rules (when "offline") or an ellmer structured-output chat session.
ai_extract_labels(text, findings = NULL, chat = ai_chat("offline"))ai_extract_labels(text, findings = NULL, chat = ai_chat("offline"))
text |
Character vector of report text. |
findings |
Optional character vector of extra finding terms to track. |
chat |
A |
A tibble::tibble with columns finding, anatomy, polarity,
confidence, and sentence.
ai_extract_labels("No evidence of pneumothorax. Moderate pleural effusion.")ai_extract_labels("No evidence of pneumothorax. Moderate pleural effusion.")
Computes morphology, first-order intensity, 3D GLCM, 3D GLRLM, and 3D GLSZM features compliant with the Image Biomarker Standardisation Initiative (IBSI) Phase-1 specification.
ai_extract_radiomics( volume, mask = NULL, discretization = c("none", "fbn", "fbw"), n_bins = 16L, bin_width = 25, backend = c("native", "pyradiomics") )ai_extract_radiomics( volume, mask = NULL, discretization = c("none", "fbn", "fbw"), n_bins = 16L, bin_width = 25, backend = c("native", "pyradiomics") )
volume |
A 3D numeric array or a |
mask |
Optional 3D binary/integer mask array (same dimensions as
|
discretization |
Intensity discretization: |
n_bins |
Number of bins when |
bin_width |
Bin width when |
backend |
Either |
A tibble::tibble with columns family, feature, and value.
ph <- ibsi_digital_phantom() rad <- ai_extract_radiomics(ph$volume, ph$mask, discretization = "none") head(rad)ph <- ibsi_digital_phantom() rad <- ai_extract_radiomics(ph$volume, ph$mask, discretization = "none") head(rad)
Generates paired 3D synthetic CT DICOM slices (ImageType = "DERIVED\\SECONDARY\\SYNTHETIC") and multi-organ integer segmentation masks
along with a provenance manifest (maisi_manifest.json). Supports local
deterministic "phantom" synthesis or an external MONAI MAISI "http"
inference endpoint.
ai_generate_maisi( out_dir = tempfile("maisi_"), anatomy_list = c("liver", "spleen", "left_kidney", "right_kidney"), dims = c(32L, 32L, 16L), spacing = c(1.5, 1.5, 2), mode = c("phantom", "http"), endpoint = NULL )ai_generate_maisi( out_dir = tempfile("maisi_"), anatomy_list = c("liver", "spleen", "left_kidney", "right_kidney"), dims = c(32L, 32L, 16L), spacing = c(1.5, 1.5, 2), mode = c("phantom", "http"), endpoint = NULL )
out_dir |
Output directory for synthetic DICOM series and manifest. |
anatomy_list |
Character vector of target organ names (e.g.
|
dims |
Integer vector |
spacing |
Numeric vector |
mode |
Synthesis mode: |
endpoint |
Optional MAISI HTTP endpoint URL when |
An S3 "dicom_maisi_result" object with dicom_dir, mask,
anatomy_map, manifest_path, and synthetic = TRUE.
out <- tempfile("maisi_ex_") res <- ai_generate_maisi(out, dims = c(16L, 16L, 4L)) dim(res$mask) unlink(out, recursive = TRUE)out <- tempfile("maisi_ex_") res <- ai_generate_maisi(out, dims = c(16L, 16L, 4L)) dim(res$mask) unlink(out, recursive = TRUE)
Maps heterogeneous SeriesDescription, Modality, and BodyPartExamined
headers onto standardized LOINC-RSNA Radiology Playbook IDs (rpid) and
RadLex anatomical identifiers (radlex_id).
ai_harmonize_protocol(headers, chat = NULL)ai_harmonize_protocol(headers, chat = NULL)
headers |
A data frame / tibble with |
chat |
Optional |
A tibble::tibble with columns SeriesDescription, Modality,
rpid, long_common_name, short_name, body_region, radlex_id,
contrast, and confidence.
df <- data.frame(SeriesDescription = "CHEST WO CON 3.0mm", Modality = "CT") ai_harmonize_protocol(df)df <- data.frame(SeriesDescription = "CHEST WO CON 3.0mm", Modality = "CT") ai_harmonize_protocol(df)
Add a Study Volume to a DuckDB Embedding Index
ai_index_add(index, study_id, vol, metadata = list(), backend = "embeddings")ai_index_add(index, study_id, vol, metadata = list(), backend = "embeddings")
index |
A |
study_id |
Unique study identifier string. |
vol |
A |
metadata |
Optional named list of study metadata. |
backend |
Registered embedding backend name (default |
Invisibly returns study_id.
Close a DuckDB Study Embedding Index
ai_index_close(index)ai_index_close(index)
index |
A |
Invisibly returns TRUE.
Uses DuckDB vss cosine similarity when installed, falling back seamlessly
to vectorized cosine scoring for air-gapped environments.
ai_search_similar(index, query_vol, k = 5L, backend = "embeddings")ai_search_similar(index, query_vol, k = 5L, backend = "embeddings")
index |
A |
query_vol |
Query |
k |
Integer number of nearest neighbors to return (default |
backend |
Registered embedding backend name (default |
A tibble::tibble with study_id, modality, similarity, and
metadata_json, sorted by descending cosine similarity.
Runs segmentation via a registered backend ("totalsegmentator", "nnunet",
"vista3d", "medsam2", or custom registered backends), aligns the mask to
the reference dicom_volume, and optionally computes voxel-wise Shannon
entropy uncertainty and Dice agreement.
ai_segment( vol, backend = "totalsegmentator", prompts = NULL, estimate_uncertainty = TRUE, reference_seg = NULL )ai_segment( vol, backend = "totalsegmentator", prompts = NULL, estimate_uncertainty = TRUE, reference_seg = NULL )
vol |
A |
backend |
Registered segmentation backend name. |
prompts |
Optional prompt list for interactive models such as
|
estimate_uncertainty |
Logical. Compute voxel-wise entropy map
( |
reference_seg |
Optional reference |
A list of class "dicom_ai_segmentation" with seg (dicom_seg),
uncertainty (dicom_volume or NULL), probs, metrics, and
backend.
dicomtk::dicom_llm Object to dicom_ai_chat
Convert a Legacy dicomtk::dicom_llm Object to dicom_ai_chat
as_ellmer_chat(x)as_ellmer_chat(x)
x |
A |
A "dicom_ai_chat" object.
Apply Fitted ComBat Harmonisation Parameters Without Test-Set Leakage
combat_apply(fit, features, batch, mod = NULL)combat_apply(fit, features, batch, mod = NULL)
fit |
A |
features |
Numeric matrix ( |
batch |
Factor or character vector of batch labels matching |
mod |
Optional biological covariate matrix ( |
Harmonized numeric matrix of dimensions dim(features).
Estimates batch location and scale parameters on a training cohort while
preserving biological covariates (mod). Apply the resulting fit to
training or held-out test cohorts with combat_apply() without test-set
leakage.
combat_fit(features, batch, mod = NULL, eb = TRUE, ref_batch = NULL)combat_fit(features, batch, mod = NULL, eb = TRUE, ref_batch = NULL)
features |
Numeric matrix ( |
batch |
Factor or character vector of scanner/site batch labels of
length |
mod |
Optional biological covariate design matrix ( |
eb |
Logical; if |
ref_batch |
Optional reference batch name. |
An S3 "dicom_combat_fit" object.
ellmer Tool Definitions for the dicomtk.ai MCP ServerConstructs a named list of ellmer::tool() definitions exposing
header inspection, automated QC, guardrailed de-identification dry-runs,
residual-PHI verification, protocol harmonisation, and AI segmentation.
dicomtk_mcp_tools()dicomtk_mcp_tools()
A named list of ellmer ToolDef objects.
tools <- dicomtk_mcp_tools() names(tools)tools <- dicomtk_mcp_tools() names(tools)
Validates or clamps any agent-proposed overrides, options, or policy flags
against a baseline dicomtk::dicom_deid_profile() along the strict
PS3.15 join-semilattice K < C < U < D < Z < X.
enforce_non_weakening_profile( baseline, override_tags = list(), options = character(0), jurisdictions = NULL, deidentify = NULL, patient_deid = NULL, mode = c("reject", "clamp") )enforce_non_weakening_profile( baseline, override_tags = list(), options = character(0), jurisdictions = NULL, deidentify = NULL, patient_deid = NULL, mode = c("reject", "clamp") )
baseline |
A baseline |
override_tags |
Named list of tag-action overrides. |
options |
Character vector of additional PS3.15 options. |
jurisdictions |
Optional character vector of additional jurisdictions. |
deidentify |
Optional logical flag override. |
patient_deid |
Optional logical flag override. |
mode |
|
A validated (or monotonically clamped) dicom_deid_profile object.
base <- dicomtk::dicom_deid_profile("ps3.15") stronger <- enforce_non_weakening_profile(base, override_tags = list("(0008,1030)" = "X"))base <- dicomtk::dicom_deid_profile("ps3.15") stronger <- enforce_non_weakening_profile(base, override_tags = list("(0008,1030)" = "X"))
Retrieve a Registered AI Backend
get_ai_backend(name)get_ai_backend(name)
name |
Backend name (case-insensitive). |
A dicom_ai_backend object.
Returns the 4 x 5 x 4 (74-voxel ROI, 2 x 2 x 2 mm voxel spacing) IBSI Phase-1 digital phantom for standardized radiomics verification.
ibsi_digital_phantom()ibsi_digital_phantom()
A list with voxels (4x5x4 array), mask (4x5x4 integer array),
spacing (c(2, 2, 2)), and volume (list(voxels = ..., spacing = ...)).
ph <- ibsi_digital_phantom() sum(ph$mask)ph <- ibsi_digital_phantom() sum(ph$mask)
List Registered AI Backends
list_ai_backends(task = NULL)list_ai_backends(task = NULL)
task |
Optional task filter ( |
A tibble::tibble summarizing registered backends.
list_ai_backends()list_ai_backends()
A compact reference dataset of common diagnostic imaging protocols across
CT, MR, PT, NM, DX, MG, US, and XA modalities mapped to
LOINC-RSNA Radiology Playbook IDs (rpid) and RadLex anatomical region
identifiers (radlex_id).
radlex_playbookradlex_playbook
A tibble with 46 rows and 8 variables:
LOINC-RSNA Radiology Playbook identifier.
Standardized long protocol name.
Abbreviated display name.
DICOM Modality code.
Normalized anatomical region.
RadLex anatomical identifier (RID...).
Logical indicating IV/radiotracer contrast usage.
Space-delimited matching keywords.
Curated subset of LOINC-RSNA Radiology Playbook and RadLex ontology.
Registers or updates an AI model backend (segmentation, NLP de-identification, OCR, or embedding extraction) in the session registry.
register_ai_backend( name, task = c("segmentation", "nlp_deid", "ocr", "embedding"), mode = c("offline", "rest"), endpoint = NULL, predict_fn = NULL, headers = character(0) )register_ai_backend( name, task = c("segmentation", "nlp_deid", "ocr", "embedding"), mode = c("offline", "rest"), endpoint = NULL, predict_fn = NULL, headers = character(0) )
name |
Unique backend identifier (e.g., |
task |
Task category: |
mode |
Execution mode: |
endpoint |
Base URL or local URI for the backend. |
predict_fn |
Optional custom |
headers |
Optional named character vector of HTTP headers. |
Invisibly returns the registered dicom_ai_backend object.
dicomtk.ai Model Context Protocol (MCP) ServerLaunches an MCP server via mcptools::mcp_server() exposing
dicomtk_mcp_tools() over "stdio" or "http".
start_mcp_server( type = c("stdio", "http"), host = "127.0.0.1", port = 8080L, tools = dicomtk_mcp_tools() )start_mcp_server( type = c("stdio", "http"), host = "127.0.0.1", port = 8080L, tools = dicomtk_mcp_tools() )
type |
Transport type: |
host |
Host interface for HTTP transport (default |
port |
Port for HTTP transport (default |
tools |
List of |
Called for its side effect (runs the MCP server loop).