The problem of changes

I feel that the key here is to avoid making thing over-complex, as it could cause confusion and make our development slower.

Multiple declarations

I agree that multiple active (pending or ongoing) declarations for the same donor create complications and I propose that we should try to avoid them. The active declarations represent the donor's instructions for future gift aid claims. I can see two cases for multiplicity:

1. The donor has changed their mind. We can then mark the old declaration as cancelled, adding log entries that link the two declarations in case there was a clerical error and it needs to be revived.
2. They desire a complex future preference, for example declare for all the odd years 2025, 2027, 2029, but not the even ones. Case 2 hardly seems likely, and I feel inclined not to support it in order to gain a substantial simplification.

So we could mandate maximum one active declaration, and avoid the need for either of the "solutions" in the IS. When there is a potential duplicate active declaration then we allow 3 possibilities:

• discard the new entry: the data is redundant
• mark the old entry outdated: this is an updated preference
• keep both separately: they in fact represent different donors

Types of change

I feel it will help us to phrase things in terms of the standard Drupal terminology of CRUD. In the end we will define some access restrictions on entities and fields.

• Creating a duplicate fails, except for admins. When staff create a new declaration that is apparently a duplicate we could flag it up (with AJAX or as a verification error), show a comparison of the data and offer the options as above. Or in the first instance, we could just fail and show a link to the existing one.
• Update is allowed within a short time period to correct a clerical error and after that, there are access restrictions. Can change the name or address. Cannot change start date. Can cancel (set end date to 'now') but cannot set end date in the past. Maybe can also shorten or lengthen end date to a new future date, or equally I'm happy to ban these two and require create+cancel.
• Delete is not allowed except for admin.
• For the donor, we could keep the UI really simple, avoiding the concept of entity instances. We can offer fields for them to set their current preference: a yes/no tick box (other fields hidden if no); name; address; optional start/end dates. We can then handle the entity operations behind the scenes, as create/cancel/update/create+cancel.
• For all operations, except correcting a clerical error, staff must supply "evidence", typically in the form of an attachment. There is no need for evidence for the donor because they are making changes directly and personally.

The donor has changed their mind. We can then mark the old declaration as cancelled

mark the old entry outdated: this is an updated preference

I don't think we can do either of these things. The problem is that declarations will often be partially overlapping: the old declaration may have a start date earlier than the newer one, and in that earlier period it may be the necessary declaration for donations made in that period; it's only superseded in the later period.

The basic questions with cancellation is "where do we store the new end date"? On the declaration we currently think is the active one, or on all of the relevant stored declarations? I'm thinking it's safer to apply it to all of them, in case some shenanigans changes which one is active.

Changes of start date should be very rare, and are hard to reason about; I'm happy to leave that to admins to figure out manually editing all the relevant declarations.

I feel it will help us to phrase things in terms of the standard Drupal terminology of CRUD.

I've actually been skittish about this. As you say, donors shouldn't be exposed to the complicated concept of declaration instances. But for things like cancellation, I'm not sure that non-admin staff should be either. We might need some simple forms that take the necessary entity updates, but which ones are conventional entity edit forms is not yet clear to me.

Creating a duplicate fails

I don't think so. It's a new source of legitimacy to a Gift Aid claim, even if Inland Revenue decided at audit that your previous declaration was invalid for some reason, you could still be covered if this new supposed duplicate with a different evidence trail was valid.

Update is allowed within a short time period to correct a clerical error and after that, there are access restrictions.

Yes, I've wondered about whether we would end up with a mechanism like that. Not yet fully clear to me. I guess there's always scope to upload additional evidence, so maybe the access control is at the field level.

Can cancel (set end date to 'now') but cannot set end date in the past. Maybe can also shorten or lengthen end date to a new future date

I'm thinking that it may be better on GiftAidOverviewContextController to show the user's current Gift Aid declaration status, and then have buttons like "Add declaration" or "Cancel" that go to specific forms with a UI and staff guidelines appropriate for that.

Delete is not allowed except for admin

Except within short time period as you suggest above.

I don't think we can do either of these things. The problem is that declarations will often be partially overlapping: the old declaration may have a start date earlier than the newer one, and in that earlier period it may be the necessary declaration for donations made in that period; it's only superseded in the later period.

By cancelled, I mean to set the end date to now. I believe this solves the problem you refer to.

I feel it will help us to phrase things in terms of the standard Drupal terminology of CRUD.

I've actually been skittish about this.

I meant to use that terminology in our discussion as it that's what we'll eventually have to write code in. I agree we don't necessarily want to expose it to users.

I don't think so. It's a new source of legitimacy to a Gift Aid claim, even if Inland Revenue decided at audit that your previous declaration was invalid for some reason, you could still be covered if this new supposed duplicate with a different evidence trail was valid.

Fair enough. But I feel there is a balance - having too many duplicates is unlikely to add value and just makes it harder to see what's going on. So we could still benefit from a feature that flags up any matching declarations.

The basic questions with cancellation is "where do we store the new end date"? On the declaration we currently think is the active one, or on all of the relevant stored declarations? I'm thinking it's safer to apply it to all of them, in case some shenanigans changes which one is active.

I don't feel intuitively connected to your idea that there is a special active declaration, according to some rules of your devising. It seems that at best it could only apply from the perspective of a specific donation (i.e. it varies with the donation time) and at a certain moment in time (it varies over time as declarations are altered). Mostly we will want to know if at least one declaration exists, i.e. hasActiveByAnyContext(). We might also need to put a link to one of the declarations, but it could potentially be chosen arbitrarily??

In terms of your first question, if we interpret the donor's instruction to be "please don't claim any more gift aid after XXX" then absolutely we have to decrease in bulk the end date of all declarations that currently extend after that time. This isn't too serious for audit, by your own theorem: they will trust us when we are reducing what we claim. However I would not wish to increase in bulk, because there's no need and it would create an audit nightmare. Firstly we'd need to duplicate the evidence onto every declaration and even worse if HMRC declined to accept the evidence that triggered the bulk update, and we'd like to restore the original ends. So this suggests to create a new declaration when rights are granted, i.e. the date is moved later, and bulk update when rights are removed.

In terms of retraction, then I could see that as a special case to handle by a user with admin permissions.

In the case of declarations moving into a context (a online donor with existing old declarations gets associated with a CRM contact who has recently cancelled gift aid)

I wonder if this module needs to take responsibility for that case?? True, custom or contrib code could set up complex context rules. However for this module we can keep them very simple. Possibly merging of 2 contexts would need human examination of the consequences??

I feel that the main need we have from this issue is a "bulk cancel" operation which decreases the end date on all declarations in scope.

By cancelled, I mean to set the end date to now. I believe this solves the problem you refer to.

That sounds like my second solution from the IS?

I don't feel intuitively connected to your idea that there is a special active declaration, ... It seems that at best it could only apply ... at a certain moment in time... Mostly we will want to know if at least one declaration exists, i.e. hasActiveByAnyContext(). We might also need to put a link to one of the declarations, but it could potentially be chosen arbitrarily??

Good. Our thinking aligns. Ironically we each thought the other was saying the opposite!

create a new declaration when rights are granted, i.e. the date is moved later, and bulk update when rights are removed.

Yes!

In terms of retraction, then I could see that as a special case to handle by a user with admin permissions.

A good enough solution for now to be sure.

In the case of declarations moving into a context (a online donor with existing old declarations gets associated with a CRM contact who has recently cancelled gift aid)

I wonder if this module needs to take responsibility for that case??

Agreed. I think it's good to have some awareness of context edge cases in case we can sometimes avoid them. But let's not create elaborate mechanisms to handle them. Nice if we can document them.

I think we have the theoretical clarity we need, we can move on to implement.

Yes it's good, we are roughly aligned, however I am not yet satisfied we have the best option.

I agree with your point, it's good to keep some reserve declarations in case the first ones are rejected. And I agree that if someone writes a clear expression to cancel their gift aid, and we have multiple declarations then we have to cancel them all.

But still I would like to have a single ongoing declaration (subtly but importantly not the same as a single active declaration), because otherwise I feel it will become too complex and I wonder if we could achieve this by immediately cancelling the reserve ones (in a way that could be reversed if the active is rejected). Also in most cases, I wonder if we won't have a clear expression of cancellation, and instead the donor will just have happened to grant a slightly different scope on two different occasions and we can reasonably follow whichever we prefer.

Finally I am concerned that we are designing everything "bottom up" - you have identified some complex subtle cases, and I totally agree we shouldn't neglect them. However I don't yet feel I understand how the simplest possible mainline cases will look/feel - I'd like us to look into that together then afterwards come back here and I hope the answer might be more clear.

I won't write more - it seems like the right time for a discussion as per your email.

Idea/suggestions for discussion. I'm looking forward to hearing your preferences.

1) Donor interface. I feel attracted to simplifying the UI for the donor's direct self-declaration. I propose we could have a form to edit a start/end date range. This allows not exposing the entity CrUD to donors at all. Behind the scenes, the code would create a new declaration and cancel the old one. We have always maximum one active declaration. We would still need to show multiple declarations for the history.

If, for example, the donor wished to declare for all the odd years 2025, 2027, 2029, but not the even ones, then they would have to wait until at least 2026 to declare for 2027. Maybe they would forget, and we would lose a little gift aid. However this case seems pretty unlikely.

2) Staff cancellation. When the donor makes a clear expression to cancel gift aid, then we need to cancel all declarations (set their end date earlier). So we need a cancel button, with a date that defaults to now.

3) Staff new declaration. Otherwise (they never mentioned the idea that they wished to cancel, we just happen to have two declarations with different dates) then I feel we can reasonably say we have permission to do either option. The second option in the staff interface is to upload a new declaration. If there is already an active declaration, then we could write code to compare the two, and automatically cancel the inferior one (generating a clear link between the two so that this cancellation can be reversed in case HMRC subsequently reject the active one).

1 and 2 are what I was thinking, great.

I don't fully understand the first and second option in 3. I wonder about having a checkbox when staff create a new declaration with an end date "Enforce this end date on relevant previous declarations?"

Sorry "option" was my poor choice of word, I edited my previous comment. I didn't mean that I was describing 2 possible options, I meant that the form allowed staff to do two things: "cancel all" (which was already covered in point 2) and "add".

I wonder about having a checkbox when staff create a new declaration with an end date "Enforce this end date on relevant previous declarations?"

That works well for the case of changing the end date to something still in the future. However if we are setting the end date to now, then there isn't any new declaration to create. So I guess we could have your checkbox on add, still keep the "cancel all" button, but remove the date by it.

Requirement

In section 3.28 "Records to be maintained" it says that the auditor will ask to see: the declaration; notification of change of address / cancellation; copy of written statement; and "all correspondence".

3.9.1: If the charity is notified of a change in the donor’s name or address, it must keep a record of the updated information — this can be kept as an electronic copy such as a scanned document.

3.8.2: The charity should keep a record of the cancellation of a declaration, including the date of the donor’s notification.

3.8.1: The charity must maintain an auditable record of all written statements and cancellation notices.... HMRC will accept that a written confirmation has been issued where the charity can show it has issued a standard letter template in an acceptable format and provides a list of the donors to whom the letter was sent.

So we definitely have to keep a record. To what degree the we need to have actual concrete evidence (scanned document / recording / etc) is debatable in some of the cases.

Solution

So far, we have suggested putting the cancellation/change of address on the declarations, which could mean duplicating it to multiple places, perhaps adding a free-form log message to explain. This doesn't necessarily allow saving evidence.

These records all have some similarities: need to record the correspondence date, probably the method (oral/web/etc.), maybe some other dates, and the evidence (a file attachment dependent on the method). I had some ideas around this...

We could generalise the Declaration class to become instead a Gift Aid Record. It would mean adding one more field to say the type of record. This class can cover all of the above records, possibly excepting the written statement where the current code could stay.

In this new model, the records are mostly immutable; any change is handled by adding new events with the new information. We would allow editing only to correct a data-entry error. This could allow simplification to remove entity revisions.

This is interesting looking back to the IS. Regarding the 2 solutions, we do now have a "negative declaration" stored, and now we call it a cancellation, it is in the "mental model". We could even do option 1 and allow the cancellation to dynamically override declarations, however I have my doubts, because I feel when looking at a declaration I would like to know that it has been cancelled.

My current preferred idea is a hybrid of the 2 solutions: a cancellation would automatically update all declarations to alter their validity. We could separate fields for the original end date (immutable, entered in the UI) and the current end of validity (calculated, read/only in the UI).

Note that this "end of validity" is just a "shortcut" which could be recalculated at any stage (someone could write code to do this automatically). This is great for untangling data inaccuracies, such as the "complication" in the IS or if one person has ended up with 2 distinct contexts (database records) which need to be combined.