First of all, I want to apologize for the inconveniences that change may have caused you.
With 5.3.1 we addressed some memory and performance issues caused by frequently loading (large) data sources. We have intentionally changed the behavior to not load data sources anew for every test case and even expected people, who are changing data sources in code at runtime, to have to adapt their code because of that change.
krstcs wrote:they are going to revert back to automatic reload at the start of each test case as it was previously
This is true, for now (see last paragraph of this post), we are going to revert back to the behavior that data sources are reloaded at the start of each test case, because of the following reasons:
First, we thought that the change would improve performance and memory consumption, but we now realized it only helps in a small number of cases (at least in the way it is currently implemented). So it causes a breaking change and work for a number of people (like you) without providing real value too all customers.
Second, we now also realized that the change has negative effects on future possible improvements regarding memory and performance with respect to data sources. I.e. the change does allow use of data sources in a way that would immediately cause memory leaks we could not fix without breaking changes again.
So, long story short, we tried to greatly improve memory and performance, but went to far with this change. Again, we are sorry for this mistake.
However, even if we now revert the behavior back, I recommend adding the code to reload the data source whenever you change a data source from code while a test suite/test case is running. We may be forced to change the behavior in the future in order to substantially improve performance and memory consumption. Explicitly reloading the data source when you change it in code should guarantee that you have the up-to-date data available.