- Issue created by @khaled.zaidan
- 🇺🇸United States TolstoyDotCom L.A.
Could you do the warming part yourself? Just before the migrations, download the data from all the URLs to your local server and then migrate from localhost.
We have a project with a big migration (group of migrations) that runs nightly. The migration reads from a REST endpoint using the url plugin.
The issue we have is that sometime one of the requests to the API would timeout. This results in the migration failing and getting stuck. There's no obvious way to just resume the migration, meaning that the whole migration needs to restart.
N/A
Options:
- It would be good if in its simplest forms, the url plugin would just retry the endpoint a few times before giving up. Then if the API is just having a hiccup and recovers from it, then the migration can carry on.
- Maybe the ability to resume a failed migration? Although, I appreciate this might fall outside the scope of this module.
- Do some kind of warming. Load all URLs and save them locally before hand, and then the migration itself would just read from the local storage. Again, this might be outside the scope of this module, but I'm not sure... any thoughts on this?
1. Decide approach
2. Implement approach
N/A
N/A
N/A
Active
6.0
Plugins
Could you do the warming part yourself? Just before the migrations, download the data from all the URLs to your local server and then migrate from localhost.