[Parent] Store more transaction specific information on Payment entities

Here is a patch that adds fields to store card payment details and adds it to the payment view.

I considered adding an update hook to copy payment method details to payments from existing orders. However the payment methods could have been edited since then and it would be copying over incorrect information.

Realized I didn't finish the modifications to the payment view. Now displays the payment method details properly.

Putting this up for review. Has tests and upgrade hook for the added fields.

Maybe we want to create a new issue for the 3.x branch for having different payment types (credit card, giftcard, check, ACH, crypto, etc).

After talking a bit about the implementation in slack I decided to rename the fields and functions so the changes are less credit card specific and can be used for other payment types.

Examples: Paypal payment gateway could set payment_type to be whatever was used to actually pay such as Visa, ACH, PayPal Credit, Pay in 4, etc. Or if it was paid for with crypto, the payment_type would be the coin used. Credit card gateways would fill in the card type such as Visa, Mastercard, etc.

The payment_identifier would be anything that would help identify the payment method used without having to lookup transaction details on the payment provider's systems, if they still exist. This could be last 4 digits of a credit card number for credit card gateways, a wallet id, etc.

Forgive me at the following stupid questions. "Dumb questions" hour?

I was looking a bit at various issues regarding payments and credit card transactions. One of the things that has appeared relatively clear (to me) is that there is a desire to not only store more transaction specific information on the payments, but also catalog things for investigatory purposes. That said, I have a somewhat different approach to the idea of the issue...

I have been dragged, kicking and screaming no less, into the idea that the receipt is the actual thing the payment is hinged upon. The payment is one side of the transaction, but the receipt is the other. In its purest form, the term "receipt" is the acceptance/receipt of a thing, an object, a widget, whatever. So, let me ask this:

Does there _actually_ need to be more information stored in the payment table, or payment method tables?

It appears (to me) the storage mechanism for the results of a processed credit card/crypto/whatever is _actually_ the receipt not the payment. The result of the transaction processed by the payment gateway is the receipt of funds. The receipt of funds occurs through a merchant id (gateway?), a transaction number (provided by the gateway, but specific to the receipt of that particular transaction), and potentially batch number or response codes... Those are all in connection with a receipt/response from the gateway.

If the gateway fails to process the transaction (response code indicating failure/no receipt?), then there probably is no reason to generate a receipt record. Am I way off base in this?

I mentioned on slack a few days ago there may be some merit to querying the commerce_payment table in connection with the receipt table, that was to suggest the legitimate payments are only those that are receipted. However, the receipt only occurs when something is received.

If the gateway fails for one reason or another, perhaps that is a separate type of record? (Not bundle, because the receipt _is_ _the receipt_.) But something different (entity) ... something to catalog the request as a failure, thereby not an actual receipt, but a forensic tool of some sort?

Is it possible there are valid receipts and invalid receipts? If that is the case, then bundles need to be classified as valid versus invalid, or... JOIN()s need to be created from payment against two separate entities (receipt and this other thing), with which ever entity holding the corresponding record is the result of the receipt, valid or invalid...

Hmm...

Does any of this make sense, or am I just wildly off in my procedural thinking? It is a complex problem, but there has to be a way to solve it.

In commerce, the payment is the receipt. It's the record of the payment. The payment method on the other hand is something that changes over time.
Typically on a receipt (payment) you want a record of what payment method was used. Currently commerce does this with a reference to the payment method that was used.
Since the payment method is not static and can change at any moment after the payment is recorded the information about the payment method can be lost.
This issue aims to add information about the payment method to the receipt (payment) so there is a record of it at the time of payment.

Re-reading your question you keep referring to a receipt table. What is this? Is this some contrib module?

Yes. Commerce_Receipt. It is a general contract/business process that acknowledges the parties, each, received ("receipt of") their ends of the agreed bargain (order). Commerce received its payment (as opposed to a promise to pay), and the customer received their products.

The payment record is a promise to pay. It is usually then sent to a payment processor, (sometimes a physical cash register, other times an omline payment processor) which actually conducts the financial transaction, and then a receipt is generated. (Authorization code, typically in combination with a merchant id and/or batch process number.)

The receipt for the transaction comes from the gateway saying "Yup, we're good" or "Nosirree, Bob. We have to try again for X reason."

The receipt of funds, unless it is cash, needs to be verified by a third party. That is largely to ensure protection on both sides of the transaction. The payor has the ability to stop payment, and the receiver has the ability to prove the payor got their end of the agreement, and thereby are required to release payment... That is the reason for third parties...receipt of the agreed upon terms, for both sides.

Organizations, in general, do not provide things up front on a promise to pay. There is a receipt that says the organization got its money, and the customer got their purchased products or services. It is a transaction: this for that.

All of that said, payments themselves are not technically receipts, hence the reasoning for looking at commerce_receipt as the appropriate place to store receipt information. (Batch codes, auth codes, etc.)

Since the payment method is not static and can change at any moment after the payment is recorded the information about the payment method can be lost.

That is the exact reason to store the info on the receipt.

Please do not take any of this as combative, it is just how I've come to understand the processes at play.

Please, argue with me. I say that, not in jest, rather as a way to truly devise the best possible solution. Whenever I say "argue with me," it is out of respect for you and your knowledge, expertise, and experience.

The payment being the receipt does offer some beauty in its simplickty. At the same time, it does remove some possibilities.

The ability for organizations to do projected cash flow or forecasted liabilities is hinged upon the receipt record. Consider projected revenues based on...what? The receipt is what allows projections versus imaginations.

If the promise to pay is not a thing, then it simultaneously removes the ability to do things like purchase orders. It constricts an organization to always requiring the receipt of funds to recognize a legitimate contract, even when the payment has previously been agreed.

There are modules, like Invoice → and Invoice Payment → . It appears those exist to handle the aforementioned issues. The promise to pay versus the receipt of funds.

I need some more clarification here. Are you arguing that details about the payment method should not be stored on the payment? Or that a more robust receipt storage is missing in commerce?

There currently is no receipt entity in commerce. There's the order which is also treated as the invoice or maybe receipt of what was agreed upon. The payment is attached to the order. That records information about how it was paid for. Usually a third payment gateway stores a transaction on their end with information about the payment made. If it's an internal payment gateway (such as manual payment for example) then the payment entity itself is the receipt of payment. The payment is designed to be the record either directly or in reference to an external record.

The goal of this issue is to store more information about the payment transaction on the payment itself (easy) instead of having to retrieve the information from the 3rd party payment processor (complicated).

I need some more clarification here. Are you arguing that details about the payment method should not be stored on the payment? Or that a more robust receipt storage is missing in commerce?

TL;DR: I, personally, believe the receipt should be separate from the payment. I also believe there should be a more robust storage of receipts in Commerce. The reason, in my view, for the separation is that the receipt is both the acceptance and the offset record. It turns an asset into a liability and vice versa, depending on the View and Context. (View and Context used in Drupal's terminology.)

Let's say you're a small e-commerce site, and your niche is specialty products. Customer John Doe logs in and creates an order with a promise to pay because your specialized products are costly to make. You don't have a lot of inventory. The promise to pay is an asset in a forecast/projection model, but becomes a liability once it's accepted/receipted. Simultaneously, the specialty product sitting on the shelf is a liability until it's received by John Doe, then it becomes an asset. Those are the payment vs receipts in the context of a business.

It all depends on context and views. (No pun intended)

Many will say, "You've got that backwards, the things sitting on your shelf are assets!" My response is, "Are they?" It seems to me, if the context is business then the product sitting on the shelf is a liability ... because it's been paid for/manufactured and it's still sitting on the shelf and not not being enjoyed by a buyer. If the context of that exact same situation is in appraisal (instead of business), then the products on the shelf are assets (because they've been paid for/manufactured), until they become liabilities upon receipt by an insurance company covering a disaster.

Now, before ripping me apart...

The goal of this issue is to store more information about the payment transaction on the payment itself (easy) instead of having to retrieve the information from the 3rd party payment processor (complicated).

There is beauty in simplicity. I mean that. No sarcasm. Simplicity makes it easier for small businesses to run, because there is less complexity. There is a reason Quicken and Quickbooks are a thing. It does not (necessarily) get into the nitty, gritty, nuance as it caters to the vast majority of use cases. That is 1000% understandable, laudable, achievable, and a directed goal. I am not arguing Commerce should go one way or the other. I am simply trying to provide a little more information as to why complexity in the backend is sometimes necessary to garner the most market share. If it is easy enough to use for mom-and-pop shops, but powerful enough for fintech and Fortune 500s...

In either case, _I_ would be hesitant to mess with the existing payment entity. The presumption that the payment is the receipt removes the ability to promise to pay and forecasting/projections (both assets and liabilities). Projections and forecasting are often necessary to secure outside financing/investment, or simply budgeting based on knowing what contracts (read: commerce_order) are ready to go, but have not been accepted yet. The presumption of promise to pay not being a thing may not work for organizations that work in arrears. Sometimes that's non-profits, sometimes it's government agencies, sometimes it's special districts, etc.

When all is said and done, I may just be full of wind, and may be so far off base that there is a necessary disregard for the commentary/noise.