Step 1c — subscription behaviour after the split - #784
Conversation
895d18a to
f50f2fb
Compare
|
Claude notes: The riskiest, and the checklist is longest. The core thing to prove is the one-invoice behaviour, in Stripe's dashboard rather than in squarelet:
Then the cancellation matrix: cancelling one line of two leaves the other billing and leaves both items on Stripe until period end; cancelling the only line sets cancel_at_period_end; resubscribe reverses each. And one interface check I'd not skip: diff an organization's OIDC entitlements payload before and after the migration. That's the contract MuckRock and DocumentCloud consume. It should be byte-identical — same entries, same resources, same quantity — but it's the thing that breaks other teams if I'm wrong. Two caveats on the checklists: they assume Stripe test keys in the preview environment, and #782/#784 both assume consolidate_stripe_products has been run there first, since the backfill needs the prices to exist. |
f50f2fb to
a3df14a
Compare
a3df14a to
135cb72
Compare
135cb72 to
05cdb2e
Compare
05cdb2e to
f84a278
Compare
0155b31 to
09579c2
Compare
09579c2 to
525fa0c
Compare
525fa0c to
f26dcab
Compare
f26dcab to
2cc708d
Compare
Removing one plan from a multi-plan subscription dropped it from Stripe immediately, while removing an org's only plan let them keep it until the period they had paid for ran out. Same button, different outcome depending on how many other plans the customer happened to have. SubscriptionItem now carries `cancelled` and `cancel_at`, mirroring Subscription. Cancelling a line flags it and leaves it billing; the line still grants access, and `uncancel()` reverses it. Stripe has no per-item cancel_at_period_end, so `restore_organization` is what enforces it: once `cancel_at` arrives the line is removed from the Stripe subscription with proration suppressed, since it has already been paid for through the end of the period. That sweep runs at 00:05, comfortably ahead of the renewal invoice for any subscription whose period ends later in the day. A subscription whose period ends within the first hour after midnight could still be invoiced for a line that was due to go; see the note in the plan. test_subscription.py is split into TestSubscription and TestSubscriptionItem, which also keeps either class under the public method limit. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Two things the test suite could not see, because neither path had coverage. `get_subscription_next_date()` read `stripe_subscription` off the object it was handed, but both callers hand it a SubscriptionItem and that property moved to the parent. Manage Subscriptions and Cancel Subscription raised AttributeError on render. There are two copies of the function; the unused one in organizations/views is fixed too rather than left as a landmine. Cancelling a whole subscription left its lines unflagged, so a customer who cancelled their only subscription still saw it listed as active -- the templates iterate lines, not subscriptions. Subscription.cancel() now flags every line it carries and uncancel() clears them, and the renewal sweep skips lines whose whole subscription is already going, so it never tries to remove them from Stripe individually. Not addressed here, both pre-existing: get_subscription_next_date makes one live Stripe call per line on every page render, and payments/views sets `cost` from `plan.base_price`, ignoring price_per_user. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Cancel and resubscribe are already per-line -- both views key on a SubscriptionItem pk -- but the model behind them got two cases wrong. `cancel()` asked whether the subscription had one item, not one *active* item. Cancelling two lines one at a time therefore flagged both and never cancelled the subscription, leaving Stripe to renew it; the sweep would then have tried to delete the subscription's only remaining line, which Stripe rejects. It now counts lines that are not already cancelled, so the second cancel is recognised as the last one. `uncancel()` cleared the line's own flag even when the whole subscription was cancelled, so a customer could resubscribe, see the line listed as active, and still have Stripe stop it at period end. Reviving a line on a cancelled subscription now revives the subscription and every line on it, which is symmetric with cancelling the last line taking them all. Also removes a stray `print(subscription)` from the resubscribe view. Packs still cancel independently of the base plan they extend, which is wrong but needs a relationship the schema does not have yet -- written up under "Add-on Packs — Follow-up Work" in the plan, together with the missing pack-quantity UI. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Manage Subscriptions renders one row per subscription line and asked Stripe for the period end on every one, so a customer with three lines made three live API calls just to draw the page. `current_period_end` is already cached on Subscription, kept current by the webhook and verified by `audit_subscriptions`, so a new `next_date` property reads it directly and the page costs one query. The list is select_related on subscription and plan, so walking to the parent adds nothing. The conversion is the part worth care: the cached value is UTC, and a period ending just after midnight UTC is still the previous evening locally. `localtime()` keeps the date the customer sees the same as before; there is a test pinning exactly that case. Both copies of `get_subscription_next_date` are gone -- the one in organizations/views was dead already. Also restores a comment that a bulk rename had turned into "SubscriptionItem views". Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Both commands walked SubscriptionItem and did select_related on "organization", which is a property now -- so both raised FieldError on their first query. Worse, both filtered `subscription_id=None`, which still runs but now refers to the FK rather than the Stripe id, so the "no Stripe ID" report would have come back empty while real records went unreported. Neither command had any test coverage. Both are subscription-level operations: subscription_id, stripe_status and current_period_end all describe the Stripe subscription rather than any one line on it. They now walk Subscription and prefetch its items, and report every plan on a subscription instead of a single slug. The audit's comparison is the part that actually needed rethinking. It read items.data[0] throughout, which silently compares the wrong line as soon as a subscription bills more than one. _compare_items now matches each local line to its Stripe item by stripe_item_id and reports lines missing from Stripe, quantity and price drift per line, and -- the case that costs money -- Stripe items we have no local record of at all. Three bugs found while verifying against the dev database: - sync_subscriptions caught InvalidRequestError, but retrieve() swallows it and returns None, so a subscription deleted on Stripe crashed with AttributeError instead of being reported - it cached stripe_status but saved only current_period_end, dropping every status correction it made - iterator() after prefetch_related() needs a chunk_size; both call sites materialise the queryset anyway Also removes imports orphaned by deleting get_subscription_next_date. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The container sets HOME to /app, which is the repo bind-mount, so anything writing to $HOME lands in the working tree. .cache and .ipython/ are already ignored for this reason; the Stripe library adds .config/stripe/telemetry_id the first time a command talks to Stripe. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Folds 0084's related-name AlterFields into the rename that caused them -- they were a second generated pass, and nothing here is deployed -- then renumbers so the chain is 0083 rename, 0084 parent, 0085 cancellation. Rehearsing the down path against a scratch database showed it did not work, in three separate ways: - Reversing re-added SubscriptionItem.organization as NOT NULL before split_back_out could populate it. The column is now relaxed in 0083, one migration earlier, so reversing re-adds it nullable, fills it, and restores NOT NULL in a transaction of its own. Postgres will not ALTER a table in the same transaction that just wrote to it, which is why the relaxation cannot simply sit next to the data move. - split_back_out deleted every Subscription row, queueing deferred foreign-key triggers that blocked the DDL following it. Reversing CreateModel drops that table anyway, so the delete was redundant. - The remaining writes left their own pending triggers, so the reverse now ends with SET CONSTRAINTS ALL IMMEDIATE to flush them. Verified by seeding a subscription at head, migrating back to 0082 and forward again: organization, subscription_id, cancelled, interval, collection_method and quantity all survive the round trip. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
An organization holds one subscription per billing shape, so two annual plans share a subscription. Subscription.auto_renew was `all(item.plan.auto_renew ...)`, which meant adding a single auto_renew=False plan would set cancel_at_period_end on the whole subscription and cancel the renewing lines along with it. The subscription now renews while any line still wants to. A non-renewing plan instead flags its own line at start, with cancel_at taken from the period end, so the existing sweep drops just that line -- the same path a line the customer cancelled takes, which also means Resubscribe already reverses it. A subscription ends only once every line has stopped. The sweep will not strip a subscription's last line: Stripe rejects removing the only item, and in that case it is already cancelling the subscription itself at period end. Latent until now, since no plan sets auto_renew=False, but it would have fired the first time one did. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Both values it showed were reading attributes that do not exist. `organization.plan` was removed from the model years ago and `subscription.update_on` was never on a subscription at all; Django resolves a missing attribute to the empty string rather than raising, so the page told every paying customer their plan was "Free" and rendered "Subscription ends on" with no date. Both predate this branch, but the split is what makes them fixable: `get_plans()` lists what an organization actually holds, and `cancel_at` now exists on the line. Listing plans rather than one also matches what the split allows -- an organization can hold a tier and a pack at once. The block moves to its own include so it can be tested. It sat above a crispy form that talks to Stripe, which is why nothing covered it and why both attributes could rot unnoticed. Also stops the view picking an arbitrary "first" line for the ends-on banner: it prefers a line that is actually ending, so the banner no longer depends on plan ordering. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
**Plan changes stopped reaching Stripe in two directions.** Master's modify() branched three ways -- free to paid started a Stripe subscription, paid to free deleted one, paid to paid modified it. The split collapsed all of that into stripe_modify(), which no-ops without a Stripe subscription. So an upgrade off a free plan granted paid access and never billed for it, and a downgrade to a free plan left the customer being charged. The second direction is the worse one and the review did not name it. Subscription.sync_to_stripe() now holds all three transitions, and modify() goes through it. **A free line was described to Stripe.** make_stripe_plan skips free plans, so naming one referenced an object that does not exist -- failing the whole call, including the paid lines alongside it. stripe_items drops them, via a new SubscriptionItem.is_free. **stripe_modify revived pending cancellations.** It cleared `cancelled` on every call and sent cancel_at_period_end=not auto_renew, so touching any line on a cancelled subscription un-cancelled it, locally and on Stripe, without anyone asking. Cancel only what was cancelled: the flag belongs to cancel(), uncancel() and the webhook, and the Stripe call now sends `cancelled or not auto_renew`. **The webhook did not cascade.** A cancellation scheduled outside our own flow -- the Stripe dashboard -- updated the parent and left every line saying it would renew. The UI lists lines. **A failed Stripe call left the line behind.** start() created the row first with nothing to undo it; both are in a transaction now. The row has to exist before the call, since it is what stripe_items describes, so the survivable direction is the other one: Stripe succeeding and the commit failing leaves something we retry into. Also: the already-subscribed check and the admin's "Will Renew" column both still keyed on subscription-level cancellation, and the cancel-at-period-end pair is now written in one place rather than four. Not fixed here, deliberately. UpdateSubscription still assumes one plan per organization -- that is the add-on pack UI already logged separately, and building it inside this PR would be a second change. And the 0084 data migration keeps its per-row queries: 250 rows, once, during a deploy that takes seconds. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`cancelled` and `cancel_at` are one fact stored in two fields: something stops at a date, or it does not. Five places wrote the pair out by hand, and two of them wrote only half of it. That matters more than it sounds, because the nightly sweep deletes anything `cancelled` whose `cancel_at` has arrived *or is null* - so a flag set without a date does not mean "pending", it means "delete tonight", subscription and every line with it. `Cancellable` now owns the pair. Both models mix it in; nothing outside it assigns either field. `push_cancellation_to_items` replaces the four hand-written `items.update(cancelled=..., cancel_at=...)` calls and copies from the subscription rather than restating the values, so a parent and its lines cannot be written to disagree. Three bugs fall out of the two half-writes: - Downgrading to free while a cancellation was pending cleared the date and left the flag. The sweep then deleted the subscription the customer had just moved onto, lines and all. - `SubscriptionItem.modify` left the old plan's cancellation in place, so the line the customer had just chosen was dropped on a date that belonged to the plan they left. The cancellation is now re-derived from the new plan, the way `start` does it, unless the whole subscription is cancelled - reversing that is `uncancel`'s job, not a plan change's. - `Subscription.start` set the date and left the flag false for a subscription of entirely non-renewing plans, disagreeing with both `stripe_modify` and the webhook, which set it. Also moves the one-off cancellation in `SubscriptionItemQuerySet.start` inside the transaction that creates the line it describes. Committing the line without it leaves a plan that was meant to bill once renewing forever, with nothing to sweep it. The first two have regression tests, checked against the pre-fix code. The transaction move has none: there is no observable difference without simulating a database failure mid-block. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0151rwxtvLg1vKEiz7XsJfzf
48a8310 to
61fd43a
Compare
Two of the six findings left from the second review held up; the other four did not. Reasoning for all six is in the GTD findings file. `audit_subscriptions._compare_items` compared every line against Stripe, including the free ones `stripe_items()` deliberately drops - naming a Plan that has no Stripe counterpart fails the whole call. A free line therefore had nothing to match and was reported "missing on stripe" on every run, on every subscription carrying one, for good. Skipped now, but only when the line also carries no Stripe item id: a free line that *is* on Stripe is real drift, and surfacing that is the point. `get_wix_labels_for_user` prefetches `organization__subscriptions__plans` and then calls `get_plans()`, which builds a fresh queryset and so cannot read the prefetch. The prefetch was paid for and ignored, and the query it was meant to save ran once per membership anyway. `get_plans()` cannot be made prefetch-aware - callers filter and slice what it returns, so it has to stay a queryset - so this adds `prefetched_plans()` alongside it, which walks the relation and returns a list. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0151rwxtvLg1vKEiz7XsJfzf
`0084` moves `cancelled`/`cancel_at` off the line and onto the parent, which is where they belong: one Stripe subscription, one cancellation. `0085` adds them back at the line level for the per-line cancellation this PR introduces - at their defaults, with nothing copying the parent's values down. The two run in one deploy, so an organization part-way through a cancellation ends up with a parent that says it is ending and lines that say they renew. Billing stays correct throughout - the nightly sweep and Stripe both act on the parent - but `manage_subscriptions.html` iterates `subscription_items` and reads the line, so the customer is shown "Renews on" for a plan that is ending, with a button offering to cancel it again, for the whole window between deploy and period end. Neither review caught this because each migration is right on its own; the gap is only visible across the #809/#784 boundary, which is another reason the two have to be read as a pair. `0085` now backfills, establishing the same invariant `push_cancellation_to_items` maintains at runtime. The two migration tests covering it were confirmed failing without it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0151rwxtvLg1vKEiz7XsJfzf
Adding a plan to any organization that predates the split fails: A new item with Price squarelet_plan_organization can't be added to this Subscription because an existing Subscription Item si_... is already using that Price. The line being rejected is not the one being added - it is the *existing* line, sent back to Stripe without its id. `stripe_items(include_ids=True)` omits the id it does not have, and a spec with no id is how you ask Stripe to add a line rather than update one. Every migrated line is in that state. `0084` adds `stripe_item_id` with an empty default, and the data migration has nothing to fill it from: the old schema had one Stripe subscription per row and no per-line id at all. So the first modification of any existing subscription is rejected - which is every current customer, the first time they change a plan. `sync_stripe_item_ids` learns the ids by matching on Price, which is safe because a subscription cannot hold the same Price twice. `stripe_modify` now calls it *before* describing the lines rather than only after, so a subscription heals as it is touched. That costs nothing: `stripe_subscription` is a cached_property the surrounding check has already fetched. `backfill_stripe_item_ids` does the same across every affected subscription in one pass, so they are fixed at deploy time rather than one customer at a time. Belongs in the runbook under release 2. This has to live here, not further up the stack. #800 already carries a version of the write-back for lines Stripe creates for us, but the runbook releases these one at a time and promises each is safe to sit indefinitely - and between release 2 and release 4, this is what breaks. Splits the cancellation tests into their own module; the subscription one had grown past a thousand lines. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0151rwxtvLg1vKEiz7XsJfzf
Stacks on #809, which carries the rename and the data model. This half is everything that changes what the system does — 25 files, no mechanical repointing.
What changed
0085adds the columns.sync_to_stripe()— the three transitions a plan change can cause, in one place: entirely free (delete the Stripe subscription), newly paid (create one), still paid (modify).Squash the split migrations and make them reversiblefinalises0083/0084from Step 1b — the Subscription / SubscriptionItem data model #809 and adds0085. The migrations that ship are the ones in this PR.The eight review findings
All in
Fix eight review findings on the subscription/item split, the last commit — worth reading on its own:modify()branched three ways; the split collapsed it intostripe_modify(), which no-ops without a Stripe subscription. So upgrading off a free plan granted paid access and never billed, and downgrading to a free plan left the customer being charged. The second direction was not in the review.stripe_modifyclearedcancelledon every call, silently reviving pending cancellations locally and on Stripe.Not fixed, deliberately:
UpdateSubscriptionstill assumes one plan per organization — that is the add-on pack UI already logged separately. And0084's data migration keeps its per-row queries: 250 rows, once, during a deploy that takes seconds.A second review found three more, in that same last commit
cancelled/cancel_atwriting out in four places, not caught the first time, kept producing new half-written pairs — including one introduced by the fix above. Rather than patch a fourth site, the pair now has one owner:Cancellable, mixed into bothSubscriptionandSubscriptionItem, withmark_cancelled()/clear_cancellation()as the only place either field is assigned. SeeGive the cancellation pair one owner.start()does it.SubscriptionItemQuerySet.start()was written outside the transaction that created the line. Fixed by moving it in.QA
0085backfills the line-level flags from the parent — without that, every customer mid-cancellation at deploy time is told their plan renews until the period ends.🤖 Generated with Claude Code
https://claude.ai/code/session_0151rwxtvLg1vKEiz7XsJfzf