tm0/flow
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Complete action runtime rewrite

Browse Source
This commit is contained in:
Tomas Mirchev
2026-05-14 13:40:11 +03:00
parent 3503d81b06
commit b05d3589b7
41 changed files with 700 additions and 899 deletions
Download Patch File Download Diff File Expand all files Collapse all files

255
docs/refactor-plan.md
View File

@@ -1,23 +1,23 @@
# Flow CLI Refactor Plan
# Flow CLI Refactor Status
> Based on code review (2026-03-22) and architecture discussion.
> Based on code review (2026-03-22), architecture discussion, and the current implementation.
> Spec: `docs/superpowers/specs/2026-03-16-flow-architecture-redesign.md`
## Current State
The rewrite from the original "vibe-coded" codebase is **largely complete**. The four-layer
architecture (core -> domain -> services -> commands) is in place, 303 tests pass, and the major
structural problems from the old codebase (duplicated code, monkeypatching, dead modules, singleton
abuse) have been resolved.
The action-runtime rewrite is implemented. `cli.py` is a thin Typer adapter, `flow.app` owns
application orchestration, domain modules keep pure planning and resolution logic, and
executor-managed mutations are represented as action plans before they reach runtime adapters.
What remains is a second pass: finishing incomplete features, unifying the repo abstraction, and
trimming redundant commands.
The old structural problems from the original codebase (duplicated flows, monkeypatching, dead
modules, singleton-style runtime access) have been removed from the active command paths. The
remaining refactor work is deferred cleanup, not a blocker for the action-centered architecture.
---
## Agreed Command Surface
## Command Surface
From the architecture discussion. This is the target.
This is the implemented command surface.
```
flow remote enter <target> # Host only. SSH+tmux into VM.
@@ -40,16 +40,16 @@ flow dotfiles edit <package> # Pull -> $EDITOR -> commit+push.
flow dotfiles repos list # List ALL managed repos (dotfiles + modules).
flow dotfiles repos status [--repo=x] # Git status for one or all repos.
flow dotfiles repos pull [--repo=x] # Pull one or all repos.
flow dotfiles repos push [--repo=x] # Push one or all repos.
flow dotfiles repos pull [--repo=x] [--dry-run]
flow dotfiles repos push [--repo=x] [--dry-run]
flow setup run [--profile p] # Bootstrap a machine.
flow setup run [profile|--profile p] [--dry-run] [--var KEY=VALUE]
flow setup list # List profiles.
flow setup show <profile> # Show profile plan.
flow packages install <name...> # Install packages from manifest.
flow packages install [name...] [--profile p] [--dry-run]
flow packages list [--all] # List packages.
flow packages remove <name...> # Remove packages.
flow packages remove <name...> [--dry-run]
flow projects check [--fetch] # VM only. Git health across ~/projects.
flow projects fetch # Fetch all project remotes.
@@ -60,8 +60,9 @@ flow projects summary # Quick status overview.
- `dotfiles` -> `dot`
- `packages` -> `package`, `pkg`
- `projects` -> `project`
- `setup` -> `bootstrap`
- `projects` -> `project`; `flow sync` -> `flow projects check --fetch`
- `setup` -> `bootstrap`, `provision`
- `remote enter` -> `enter`
- `dev attach` -> `dev connect`
- `dev rm` -> `dev remove`
- `dotfiles repos` -> `dotfiles repo`
@@ -72,7 +73,7 @@ flow projects summary # Quick status overview.
- `--quiet` / `-q`
- `--no-color`
### Commands removed (vs current implementation)
### Commands Removed During Refactor
| Removed | Reason |
|---------|--------|
@@ -83,7 +84,31 @@ flow projects summary # Quick status overview.
| `dotfiles modules list` | Replaced by `dotfiles repos list` |
| `dotfiles modules sync` | Replaced by `dotfiles repos pull` |
### Key design decision: modules are repos
## Action-Centered Architecture
The runtime boundary is `flow.actions`.
- `ActionPlan` is the unit of execution. It can contain high-level `DomainAction` entries and direct
`PrimitiveAction` entries.
- `DomainAction` records intent from a domain such as dotfiles, packages, repos, remote targets,
containers, completion, or setup.
- `expand_actions()` converts domain actions into primitive actions. Some domains supply already
expanded primitive plans when the service has concrete runtime arguments.
- `PrimitiveAction` is the canonical executor input for filesystem, process, git, download,
archive, container, and tmux operations.
- `ActionExecutor` owns dry-run output, append-only JSONL audit logging, rollback stack management,
rollback barriers, and dispatch into `SystemRuntime`.
App use-cases construct plans and pass them to the executor for action-backed commands. Direct
runtime calls are limited to explicit interactive boundaries such as attaching to tmux or entering a
container shell. Domain modules stay free of I/O where the current implementation has pure
resolution/planning functions.
Rollback is best-effort and explicit. Actions default to `rollbackable`; external boundaries such
as shell commands, remote sessions, and non-reversible git/container operations use `barrier` or
`none` policies.
## Key Design Decision: Modules Are Repos
The `_module.yaml` files define external git repos that provide content for dotfiles packages.
These module repos are **not** git submodules -- they are regular git clones managed by flow.
@@ -97,185 +122,59 @@ needed (dotfiles repo is named `dotfiles`, module repos are named by their packa
---
## 1. Unified Repos Abstraction
## Completed Work
This is the most impactful change. Currently:
- `repo_status/pull/push` operate only on the dotfiles repo
- `sync_modules` handles module repos separately
- `list_modules` is a standalone method
### Unified Repos Abstraction
**Target**: a single `_discover_repos() -> list[RepoInfo]` that returns all managed repos, and
repo commands iterate over them.
`RepoInfo` with `module_ref` is the canonical repo model. `_discover_repos()` finds the dotfiles
repo and module repos, and `repos list/status/pull/push` iterate that single collection with an
optional `--repo` filter. `dotfiles init` uses the same pull-or-clone flow.
### 1.1 Add `RepoInfo` model
### Command Trimming
```python
# domain/dotfiles/models.py
@dataclass(frozen=True)
class RepoInfo:
name: str # "dotfiles" or module package name (e.g. "nvim")
path: Path # Local clone path
remote: str # Remote URL
is_module: bool # False for dotfiles repo, True for module repos
```
Removed redundant dotfiles commands:
### 1.2 Implement `_discover_repos`
- `dotfiles sync`: use `dotfiles repos pull` plus `dotfiles link`
- `dotfiles relink`: `dotfiles link` is idempotent
- `dotfiles undo`: use `dotfiles unlink`
- `dotfiles clean`: broken symlink repair is part of link planning
- `dotfiles modules list/sync`: use `dotfiles repos list/pull`
In `DotfilesService`: walk packages to find `_module.yaml` files, build `RepoInfo` for each
module repo, plus one for the dotfiles repo itself.
### Feature Completion
### 1.3 Refactor repo commands
- `dotfiles edit`: pull -> `$EDITOR`/`$VISUAL` -> scoped `git add` -> commit+push, with
`--no-commit` to skip auto-commit/push.
- `dotfiles status`: module info, link health, and package filtering.
- `dotfiles repos list`: all managed repos with name, type, local path, and clone status.
- `--no-color`: global flag added to `cli.py`.
- `--dry-run`: supported by dotfiles link/unlink, repos pull/push, packages install, setup run,
remote enter, and dev create.
Replace `repo_status`, `repo_pull`, `repo_push` with methods that iterate `_discover_repos()`,
filtered by `--repo`. Add `repos_list`.
### 1.4 Remove `dotfiles modules` subcommand group
Delete `modules list` and `modules sync` subparsers. Remove `sync_modules`, `list_modules`
methods. Remove from completion candidates.
### 1.5 Update `dotfiles init`
`init` should clone the dotfiles repo, then discover `_module.yaml` files and clone all module
repos. Currently it calls `sync_modules()` -- this should call `repos_pull()` instead (which
pulls/clones all repos).
**Files**: `models.py`, `services/dotfiles.py`, `commands/dotfiles.py`, `commands/completion.py`.
---
## 2. Remove Redundant Commands
### 2.1 Remove `dotfiles sync`
Currently does `git pull --ff-only` + `sync_modules` + optional relink. After the repos
unification, this is just `repos pull` + `link`. No need for a dedicated command.
### 2.2 Remove `dotfiles relink`
Currently calls `link()`. `link` is already idempotent -- calling it again reconciles state.
### 2.3 Remove `dotfiles undo`
`unlink` is the inverse of `link`. The backup/undo transaction machinery (`_save_backup`,
`_load_backup`, `_backup_path`) can be deleted.
### 2.4 Fold `dotfiles clean` into `dotfiles link`
`link` should detect and remove broken symlinks as part of reconciliation, not require a separate
`clean` step. Modify `plan_link` in `domain/dotfiles/planning.py` to include broken link removal
in its plan.
**Files**: `services/dotfiles.py`, `commands/dotfiles.py`, `commands/completion.py`,
`domain/dotfiles/planning.py`.
---
## 3. Previously Incomplete Features -- DONE
### 3.1 `dotfiles edit` -- DONE
Implemented: pull -> `$EDITOR`/`$VISUAL` -> scoped `git add` -> commit+push.
Flag: `--no-commit` to skip auto-commit/push.
### 3.2 `dotfiles status` -- DONE
Enhanced: shows module info (`branch:main`), link health (`ok`/`broken`/`not linked`),
package name filtering via positional args.
### 3.3 `dotfiles repos list` -- DONE
Shows all managed repos (dotfiles + modules) with: name, type, local path, clone status.
---
## 4. Spec Deviations
### 4.1 `--no-color` global flag -- DONE
Added to `cli.py`.
### 4.2 `--dry-run` coverage -- DONE for dotfiles
`repos pull` and `repos push` now have `--dry-run`. Remaining:
| Command | Has it | Should have |
|---------|--------|-------------|
| `dev stop` | No | Consider |
| `dev rm` | No | Consider |
### 4.3 Improvements over spec
### Improvements Over Spec
These are correct deviations -- the implementation improved on the spec:
- `core/containers.py` + `core/tmux.py` extracted as adapters (spec had them inline)
- `adapters/containers.py` + `adapters/tmux.py` extracted as adapters (spec had them inline)
- `core/config_parse.py` + `core/yaml.py` extracted for config parsing
- `SystemRuntime` extended with `.containers` and `.tmux` fields
- `SystemRuntime` extended with containers, tmux, download, and archive runtime fields
- `flow.actions` extracted as the canonical execution layer instead of leaving mutation dispatch in
individual app use-cases
---
## 5. Code Quality (done)
## CI
These were fixed in this session:
The GitHub Actions workflow is split into two jobs:
- `FakeRunner` consolidated from 3 test files into `tests/fakes.py`
- `services/dotfiles.py` now uses `flow.core.yaml.load_yaml_file` instead of raw `yaml`
- `ContainerRuntime.binary` no longer eagerly validates PATH for explicit modes
- `unit`: installs dependencies and runs `pytest tests/ -v --ignore=tests/e2e`
- `e2e`: verifies Docker is available, sets `FLOW_RUN_E2E=1`, and runs `pytest tests/e2e/ -v`
---
## 6. Future (defer)
## Optional Future Work
### 6.1 Bootstrap as orchestrator
These are optional refinements, not blockers for the action-centered rewrite.
The spec envisions bootstrap as a pure orchestrator over packages + dotfiles + setup modules.
Current implementation works but has its own package resolution logic. Defer until dotfiles and
packages domains are fully stable.
### 6.2 Global `--dry-run`
### Global `--dry-run`
If per-command `--dry-run` becomes a maintenance burden, promote to a global flag in `cli.py`.
---
## 7. Execution Status
All phases complete. 315 tests pass, 0 failures.
### Phase A: Unify repos + trim commands -- DONE
1. `RepoInfo` model with `module_ref` field
2. `_discover_repos()` finds dotfiles + module repos
3. `repos_list/status/pull/push` iterate all repos with `--repo` filter
4. `repos list` subcommand added
5. `dotfiles modules` subcommand removed entirely
6. `dotfiles sync`, `relink`, `undo`, `clean` removed
7. Broken-symlink handling folded into `plan_link`
8. `dotfiles init` uses unified `repos_pull()`
### Phase B: Complete features -- DONE
9. `dotfiles edit` implemented (pull -> $EDITOR -> commit+push, scoped git add)
10. `dotfiles status` enhanced (module info, link health, package filtering)
### Phase C: CLI polish -- DONE
11. `--no-color` global flag added to `cli.py`
12. `--dry-run` added to `repos pull` and `repos push`
13. Zsh completion updated for new command surface
### Phase D: Code quality -- DONE
14. Dispatch patterns: completion `complete()`, dotfiles `_git_checkout_ref`, bootstrap phases
15. `FakeRunner` consolidated to single `tests/fakes.py` (all 4 test files)
16. Bootstrap `VALID_PHASES` as single source of truth in models
17. Bootstrap models: `Any` types replaced with `ProfilePackageEntry` and `PackageDef`
18. Canonical `ssh-keys` field (removed `ssh-keygen` alias)
19. `getattr` defensive patterns removed from command handlers
20. Test coverage added: `repos_status`, `repos_push`, `repos_pull --dry-run`, status filtering,
broken symlink repair
21. README updated to reflect new command surface
### Remaining (deferred)
- Bootstrap as orchestrator (section 6.1)
- Global `--dry-run` (section 6.2)