Make endDate optional when downloading GoCardless transactions

[kyrias]

There appears to be no strong reason for explicitly limiting the end date to today, and it's preventing proper integration with Bank Norwegian whose value dates actually represent when the interest-free period for the transaction ends and is therefore in the future. (#237)

After this is merged we can then remove the corresponding field from the Actual application.

[leikoilja]

LGTM, thanks for the fix 💥
For context, here are more reasoning and details

[leikoilja]

Suggested change

Stop sending `end_date` to today when downloading GoCardless transactions.

Stop sending optional `end_date` parameter to GoCardLess which is default set to `today()` when downloading transactions from GoCardless, since it filters out credit card transactions using `valueDate` for indicating when the interest-free period for the transaction ends. [Example issue for Bank on Norwegian Sweden](https://github.com/actualbudget/actual-server/pull/237#pullrequestreview-1563979522)

[MatissJanis]

Actually.. I think I know why we would want to keep this around: it allows to paginate the results.

Currently we always get the latest transactions. But with the current API (start + end date) we could build pagination that fetches the full transaction history. It would just make multiple API calls to get it all.

[MatissJanis]

What if we made endDate optional (but still kept it around to allow the pagination implementation in UI in the future)?

[kyrias]

I was originally going to make it optional but decided against it to keep the changes only to the backend, but if we make it optional and start using it again in the future then aren't we going to run into the same issue again with Bank Norwegian? Though I guess we could 'just' allow you to keep paging into later months...

[MatissJanis]

This is how I'd imagine it working eventually:

The latest transactions are always loaded without endDate. Then we compare the oldest received transactions with what we had before. If there is something new - we start fetching the full history with multiple API calls until we hit the end of the history. If there is nothing new - we do not fetch the history and just stop.

This isn't a comprehensive solution though. And it has many question marks (how exactly do we perform the "are there more transactions that should be fetched"?). But alas it gives a vague understanding of how it might eventually work.

[kyrias]

This isn't a comprehensive solution though. And it has many question marks (how exactly do we perform the "are there more transactions that should be fetched"?). But alas it gives a vague understanding of how it might eventually work.

There simply isn't any way to do that because the API wasn't meant for that use case. If we want pagination to work properly we'd need GoCardless to actually implement a pagination feature in their API, at which point we would again not need to pass this parameter.

But I've switched this to mark the argument as optional in the only place in which the type was specified. See actualbudget/actual#1493 for the other part. Would be nice to get these two merged relatively quickly so we can get the other parts ready for Bank Norwegian support. :)

[MatissJanis]

Thanks!