Refactor djls-project by Django domain (#699)

Author: joshuadavidthomas

ARCHITECTURE.md

@@ -27,7 +27,7 @@ If you're already familiar with LSP, `crates/djls-server/src/server.rs` is a goo
 
 If you want to understand how templates get parsed, start in `crates/djls-templates/src/` — the lexer and a hand-written recursive descent parser live there.
 
-If you're curious about how the server validates tags without a Django runtime, look at `crates/djls-project/src/specs/analysis/` — it extracts validation rules from Python templatetag source purely through static analysis. If anyone's tried this before, they and their project didn't make it out with their sanity intact because I've never come across one (and who says I'll make it out with mine?). It's early and rough, but it's one of the more interesting parts of the project.
+If you're curious about how the server validates tags without a Django runtime, look at `crates/djls-project/src/templates/tags/analysis/` — it extracts validation rules from Python templatetag source purely through static analysis. If anyone's tried this before, they and their project didn't make it out with their sanity intact because I've never come across one (and who says I'll make it out with mine?). It's early and rough, but it's one of the more interesting parts of the project.
 
 If you're interested in how parsing, extraction, and project knowledge come together to give templates *meaning*, start with `crates/djls-semantic/src/lib.rs` — that's where "is this tag valid here?" actually gets answered.
 
@@ -62,7 +62,7 @@ This crate should stay boring. It wires traits to concrete state and handles loc
 
 ### `crates/djls-project`
 
-The project model. This crate owns mechanical facts about a Django project: the `Project` Salsa input, Python interpreter and search-path discovery, module resolution, Django settings extraction, template directories, template libraries, discovered template files, template origins/resolution, Python spec extraction, model graph extraction, and project refresh. It owns two tiers of source recognizers: pure recognizers in `extraction/` for Django settings values and template tag registrations, and Salsa-assisted recognizers in `specs/` for tag rules, block specs, filter arities, and Django model graphs.
+The project model. This crate owns mechanical facts about a Django project: the `Project` Salsa input, Python interpreter and search-path discovery, module resolution, Django settings extraction, template directories, template libraries, discovered template files, template origins/resolution, template tag/filter extraction, model graph extraction, and project refresh. Its internals are organized by Django domain: `settings/` handles settings-source extraction and source resolution, `templates/` handles template origins, directories, libraries, registrations, tag rules, filter arities, and symbols, and `models/` handles Django model graph extraction.
 
 `djls-project` depends on `djls-source` for filesystem/source access, but it does not depend on `djls-semantic`. That one-way boundary lets semantic analysis consume observed source facts without project discovery needing to know about template validation, scoping, or diagnostics.
 
@@ -100,7 +100,7 @@ This crate also owns:
 
 **Architecture Invariant:** extraction currently only captures constraints on *static template syntax* — argument counts and literal keyword positions knowable at parse time. Many templatetag functions also validate *runtime values* (type checks, truthiness checks on resolved variables), but those guards depend on what template variables resolve to during rendering, which the server cannot currently determine. If type inference is added in the future ([#424](https://github.com/joshuadavidthomas/django-language-server/issues/424)), some of these runtime guards may become statically evaluable — possibly as a separate analysis layer, or as an extension of the extraction pipeline itself.
 
-Project configuration, Python environment discovery, module resolution, and Python spec extraction live in `crates/djls-project/`. `Project` is a Salsa input holding the project root, interpreter path, Django settings module, resolver-owned `SearchPaths`, and manual tag-spec configuration. `djls-project` also owns the `TemplateLibraries` type that holds derived template tag library knowledge from Django settings, source files, and installed packages. Imperative refresh functions there synchronize external project state into Salsa inputs; tracked semantic queries then fuse those observed facts into validation data and other semantic facts.
+Project configuration, Python environment discovery, module resolution, and source-derived Django facts live in `crates/djls-project/`. `Project` is a Salsa input holding the project root, interpreter path, Django settings module, resolver-owned `SearchPaths`, and manual tag-spec configuration. `djls-project` also owns the `TemplateLibraries` type that holds derived template tag library knowledge from Django settings, source files, and installed packages. Imperative refresh functions there synchronize external project state into Salsa inputs; tracked semantic queries then fuse those observed facts into validation data and other semantic facts.
 
 ### `crates/djls-ide`
 
@@ -158,7 +158,7 @@ Project refresh starts from the configured Django settings module. `djls-project
 1. The `Project` input stores the project root, interpreter path, Django settings module, search paths, and manual tag-spec configuration.
 2. Template directories come from the static settings projection and are exposed as tracked project queries.
 3. Template libraries come from three source-derived places: configured `OPTIONS["libraries"]`, configured/default `OPTIONS["builtins"]`, and `templatetags` packages under installed apps.
-4. Symbols for those libraries are collected from Python source with the registration scanner in `djls-project::extraction`; `djls-project::specs` reuses the same registration facts to extract validation rules and model graphs.
+4. Symbols for those libraries are collected from Python source with the registration scanner in `djls-project::templates::registrations`; `djls-project::templates::{tags,filters}` reuse the same registration facts to extract validation rules and filter arities, while `djls-project::models` extracts Django model graphs.
 5. Search-path roots for installed packages remain high-durability source roots, so package files are reread when refresh detects that external data changed.
 
 Startup is now configuration read plus project refresh. There is no embedded inspector zipapp, no `django.setup()`, and no template-library disk cache in the server path.

CONTEXT.md

@@ -35,7 +35,7 @@ Source-based **Django Discovery** that derives **Project Facts** from source fil
 _Avoid_: introspection, runtime analysis, import-time analysis
 
 **Project/Semantic Boundary**:
-Observed source facts and lookup facts — including Python source facts and **Template Resolution** — belong to `djls-project`; project meaning — fusion, validity, availability, diagnostics, and template-domain relationships — belongs to `djls-semantic`. Only `djls-project` parses Python source.
+Observed source facts and lookup facts — including Python source facts and **Template Resolution** — belong to `djls-project`; project meaning — fusion, validity, availability, diagnostics, and template-domain relationships — belongs to `djls-semantic`. Only `djls-project` parses Python source. Inside `djls-project`, code is organized by Django domain (`settings`, `templates`, `models`) rather than by generic fact/extraction buckets.
 _Avoid_: classifying ownership by whether a type's name sounds semantic
 
 **Project Introspection**:

crates/djls-project/src/extraction.rs

@@ -1,37 +1,21 @@
 mod ast;
 mod db;
-mod environment;
-mod extraction;
+mod models;
 mod names;
 mod parse;
 mod project;
 mod python;
 mod resolve;
 mod settings;
-mod specs;
-mod symbols;
 mod sync;
 mod system;
 mod templates;
 
 pub use ast::ExprExt;
 pub use db::Db;
-pub use environment::InactiveLibraries;
-pub use environment::InactiveLibrary;
-pub use environment::inactive_template_libraries;
-pub use extraction::DjangoSettings;
-pub use extraction::InstalledAppsSetting;
-pub use extraction::RegistrationInfo;
-pub use extraction::RegistrationKind;
-pub use extraction::SettingsSource;
-pub use extraction::SettingsSourceResolver;
-pub use extraction::SettingsStarImport;
-pub use extraction::StaticKnowledge;
-pub use extraction::TemplateBackend;
-pub use extraction::TemplateDirPath;
-pub use extraction::TemplateSettings;
-pub use extraction::collect_registrations_from_body;
-pub use extraction::extract_settings;
+pub use models::ModelGraph;
+pub use models::compute_model_graph;
+pub use models::extract_model_graph;
 pub use names::LibraryName;
 pub use names::ModulePath;
 pub use names::PyModuleName;
@@ -46,54 +30,67 @@ pub use resolve::SearchPaths;
 pub use resolve::discover_model_files;
 pub use resolve::model_modules;
 pub use resolve::templatetag_modules;
+pub use settings::DjangoSettings;
+pub use settings::InstalledAppsSetting;
+pub use settings::SettingsSource;
+pub use settings::SettingsSourceResolver;
+pub use settings::SettingsStarImport;
+pub use settings::StaticKnowledge;
+pub use settings::TemplateBackend;
+pub use settings::TemplateDirPath;
+pub use settings::TemplateSettings;
 pub use settings::django_settings;
+pub use settings::extract_settings;
 pub use settings::settings_module_file;
-pub use settings::template_dirs;
-pub use settings::template_libraries;
-pub use specs::ArgumentCountConstraint;
-pub use specs::AsVar;
-pub use specs::BlockSpec;
-pub use specs::BlockSpecs;
-pub use specs::ChoiceAt;
-pub use specs::ExtractedDiagnosticConstraint;
-pub use specs::ExtractedDiagnosticMessage;
-pub use specs::ExtractedMessageArg;
-pub use specs::ExtractedMessageTemplate;
-pub use specs::FilterArity;
-pub use specs::FilterArityMap;
-pub use specs::KnownOptions;
-pub use specs::ModelGraph;
-pub use specs::RequiredKeyword;
-pub use specs::SplitPosition;
-pub use specs::SymbolKey;
-pub use specs::TagArgument;
-pub use specs::TagArgumentKind;
-pub use specs::TagRule;
-pub use specs::TagRuleMap;
-pub use specs::TemplateSymbolKind;
-pub use specs::compute_model_graph;
-pub use specs::extract_block_specs;
-pub use specs::extract_filter_arities;
-pub use specs::extract_model_graph;
-pub use specs::extract_tag_rules;
-pub use symbols::InstalledSymbolCandidate;
-pub use symbols::InstalledSymbolOrigin;
-pub use symbols::SymbolDefinition;
-pub use symbols::TemplateLibraries;
-pub use symbols::TemplateLibrary;
-pub use symbols::TemplateSymbol;
 pub use sync::RefreshData;
 pub use sync::RefreshQuery;
 pub use sync::RefreshQueryResult;
 pub use sync::apply_refresh;
+pub use templates::ArgumentCountConstraint;
+pub use templates::AsVar;
+pub use templates::BlockSpec;
+pub use templates::BlockSpecs;
+pub use templates::ChoiceAt;
+pub use templates::ExtractedDiagnosticConstraint;
+pub use templates::ExtractedDiagnosticMessage;
+pub use templates::ExtractedMessageArg;
+pub use templates::ExtractedMessageTemplate;
+pub use templates::FilterArity;
+pub use templates::FilterArityMap;
 pub use templates::FindTemplateResult;
+pub use templates::InactiveLibraries;
+pub use templates::InactiveLibrary;
+pub use templates::InstalledSymbolCandidate;
+pub use templates::InstalledSymbolOrigin;
+pub use templates::KnownOptions;
 pub use templates::ProjectTemplateFile;
 pub use templates::ProjectTemplateFiles;
+pub use templates::RegistrationInfo;
+pub use templates::RegistrationKind;
+pub use templates::RequiredKeyword;
+pub use templates::SplitPosition;
+pub use templates::SymbolDefinition;
+pub use templates::SymbolKey;
+pub use templates::TagArgument;
+pub use templates::TagArgumentKind;
+pub use templates::TagRule;
+pub use templates::TagRuleMap;
 pub use templates::TemplateDoesNotExist;
+pub use templates::TemplateLibraries;
+pub use templates::TemplateLibrary;
 pub use templates::TemplateName;
 pub use templates::TemplateOrigin;
 pub use templates::TemplateOrigins;
+pub use templates::TemplateSymbol;
+pub use templates::TemplateSymbolKind;
 pub use templates::TriedTemplateSource;
+pub use templates::collect_registrations_from_body;
+pub use templates::extract_block_specs;
+pub use templates::extract_filter_arities;
+pub use templates::extract_tag_rules;
 pub use templates::find_template;
+pub use templates::inactive_template_libraries;
 pub use templates::project_template_files;
+pub use templates::template_dirs;
+pub use templates::template_libraries;
 pub use templates::template_origins;

crates/djls-project/src/lib.rs

@@ -1,15 +1,16 @@
-pub mod extract;
-pub mod graph;
+mod extract;
+mod graph;
 
 use djls_source::File;
+pub use extract::extract_model_graph;
+pub use graph::ModelGraph;
 
 use crate::db::Db;
+use crate::models::extract::extract_model_graph_from_body;
 use crate::names::ModulePath;
 use crate::parse::parse_python_module;
 use crate::project::Project;
 use crate::resolve::model_modules;
-use crate::specs::models::extract::extract_model_graph_from_body;
-use crate::specs::models::graph::ModelGraph;
 
 /// Compute a merged `ModelGraph` from discovered model sources.
 #[salsa::tracked(returns(ref))]

crates/djls-project/src/specs/models.rs crates/djls-project/src/models.rscrates/djls-project/src/specs/models.rs renamed to crates/djls-project/src/models.rs

@@ -8,13 +8,13 @@ use rustc_hash::FxHashSet;
 use crate::ast::ExprExt;
 use crate::ast::Recurse;
 use crate::ast::walk_stmts;
-use crate::specs::models::graph::FieldName;
-use crate::specs::models::graph::ModelDef;
-use crate::specs::models::graph::ModelGraph;
-use crate::specs::models::graph::ModelKind;
-use crate::specs::models::graph::ModelName;
-use crate::specs::models::graph::Relation;
-use crate::specs::models::graph::RelationType;
+use crate::models::graph::FieldName;
+use crate::models::graph::ModelDef;
+use crate::models::graph::ModelGraph;
+use crate::models::graph::ModelKind;
+use crate::models::graph::ModelName;
+use crate::models::graph::Relation;
+use crate::models::graph::RelationType;
 
 /// Extract a model graph from Python source text.
 ///

…djls-project/src/specs/models/extract.rs crates/djls-project/src/models/extract.rscrates/djls-project/src/specs/models/extract.rs renamed to crates/djls-project/src/models/extract.rs crates/djls-project/src/specs/models/extract.rs renamed to crates/djls-project/src/models/extract.rs

@@ -247,7 +247,7 @@ pub fn templatetag_modules(db: &dyn ProjectDb, project: Project) -> Vec<PythonMo
 
     let mut modules = Vec::new();
 
-    for (module_index, module_path) in crate::settings::template_libraries(db, project)
+    for (module_index, module_path) in crate::templates::template_libraries(db, project)
         .registration_modules()
         .into_iter()
         .enumerate()