Would an automatically generated dependency graph for MIMIC-IV concepts be useful?

[Yzg18670698968]

Hi, thanks for maintaining the MIMIC code repository.

While looking through the MIMIC-IV concept build process, I noticed that the execution order of derived concepts is currently partly encoded manually. For example, make_concepts.sh handles several dependency-sensitive concepts separately, such as first_day_sofa, kdigo_stages, sepsis3, and the vasoactive-agent related concepts, so that their prerequisites are created first.

I also noticed that SQLGlot is already used in the repository for parsing and transpiling concept SQL across different dialects. This made me wonder whether it could also be used to infer dependencies between derived concepts automatically.

For example, references to tables under mimiciv_derived.* could be extracted from each concept SQL file to identify partial dependency relationships such as:

urine_output → kdigo_uo
kdigo_uo → kdigo_stages

sofa → sepsis3
suspicion_of_infection → sepsis3

These relationships could then be used to construct a directed dependency graph.

Once such a graph is available, it might be useful for:

• automatically determining a valid concept build order through topological sorting;
• detecting missing or circular dependencies;
• reducing the need to maintain dependency-sensitive execution order manually.

It could potentially also provide a machine-readable description of the dependencies among derived concepts.

Since SQLGlot is already part of the current workflow, I was wondering whether this kind of dependency analysis would be useful for the project, or whether there are aspects of the current concept-generation process that would make automatic dependency inference unreliable.

Thanks for your thoughts.

[alistairewj]

Yeah! I think it's very doable, it would be better than the current system, but I always found other things to prioritize first. It would have a nice side-effect of centralizing the dependency chain, whereas it currently exists across 3-4 files.