Skip to content

Refactor/module structure#262

Draft
bkanuka wants to merge 10 commits into
developmentseed:mainfrom
bkanuka:refactor/module-structure
Draft

Refactor/module structure#262
bkanuka wants to merge 10 commits into
developmentseed:mainfrom
bkanuka:refactor/module-structure

Conversation

@bkanuka

@bkanuka bkanuka commented Jun 10, 2026

Copy link
Copy Markdown
Contributor

Splits the three largest modules into focused packages/modules. No behavior changes — all old import paths keep working through back-compat shims.

  • collections.py split into pgcollection.py (the PgCollection implementation), dbmodel.py (data models + the abstract Collection base), filter.py (cql2→SQL filter building), and sqlhelpers.py (shared SQL helpers). collections.py now holds the catalog builders (pg_get_collection_index, register_collection_catalog) and re-exports the moved names.
  • model.py split into the models/ package (common, features, tiles); model.py is now a re-export shim.
  • factory.py split into the factories/ package (base, features, tiles, endpoints, utils); factory.py is now a re-export shim.

vincentsarago and others added 10 commits May 27, 2026 16:15
Break up the three largest modules without changing behavior; old import
paths keep working via back-compat shims.

- collections.py split into: pgcollection.py (PgCollection), dbmodel.py
  (data models + abstract Collection base), filter.py (cql2->SQL filter
  building), and sqlhelpers.py (shared SQL helpers). collections.py now
  holds the catalog builders (pg_get_collection_index,
  register_collection_catalog) and re-exports the moved names.
- model.py split into the models/ package (common, features, tiles);
  model.py is now a re-export shim.
- factory.py split into the factories/ package (base, features, tiles,
  endpoints, utils); factory.py is now a re-export shim.
@bkanuka

bkanuka commented Jun 10, 2026

Copy link
Copy Markdown
Contributor Author

@vincentsarago Would you be interested in something like this that breaks up long files? This includes commits from #260

@vincentsarago

Copy link
Copy Markdown
Member

@bkanuka thanks for this PR.

This is something we had in mind for some time (especially to support other backend than PG like duckdb).

The PR in this state is impossible to review sadly. I would prefer we finish #260 and think a bit more how we can split files. IMO it's not really necessary to split eveything (e.g the model and factory can stay as they are today IMO)

@bkanuka

bkanuka commented Jun 11, 2026

Copy link
Copy Markdown
Contributor Author

Totally understandable - a big change like this touches every line and it's a huge shock of a PR. I don't know the best way of going about this collaboratively because one way or another it's going to be a massive PR.

Do you want me to tackle just collections.py ? Or break up each move into a different commit?

I agree let's do 260 first.

@bkanuka

bkanuka commented Jun 11, 2026

Copy link
Copy Markdown
Contributor Author

One of the things I'd like to achieve in a refactor, is pullout out the cql2 filter to SQL building into a more "standalone" function. It would take in the cql2 and the geometry column and return a fully built WHERE clause (maybe it would need something else as input, I'm not at my desk right now).

My intention would be to use it outside this project for creating new derivative tables from the cql2 filter.

I'm hesitant if that would even belong here or a different project though. It could arguably live here or with cql2-rs or even it's own project.

@vincentsarago

Copy link
Copy Markdown
Member

I understand, the code base is a bit big so it might be hard to onboard newcomers but I also do want to split the code into to many interconnected sub-modules just for the sake of it.

I

One of the things I'd like to achieve in a refactor, is pullout out the cql2 filter to SQL building into a more "standalone" function. It would take in the cql2 and the geometry column and return a fully built WHERE clause (maybe it would need something else as input, I'm not at my desk right now).

Interesting, TBH I don't have a ton of free time to maintain this project right now so I would be hesitant to handle a refactor. Feel free to open an issue and then describe a work plan and maybe some pseudo code before starting a new PR 🙏 (I don't want to waste your time)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants