[Performance] Improved FLORIS interface

[MarcusBecker-GitHub]

FLORIS recently explored GPU parallelization with improved performance for large wind farms.
This begs the question, if maybe the FLORIS internal engine is better optimized to run multiple small wind farms in one simulation, rather than running multiple simulations of the same wind farm.
To this end, the following actions are proposed:

• Check if FLORIS deals better with a large number of small wind farms in one simulation than multiple simulations of one wind farm
• Add consistent performance logging to get a better overview of where the time is spent.
• Move the turbine interaction loop calculation to the wake solver. This reduces the complexity of the main loop and offers more flexibility to the wake solver, but might interfere with current code extensions.

[rafmudaf]

Hi @MarcusBecker-GitHub - The GPU solver we've prototyped in FLORIS does do better with one simulation of many turbines rather than multiple simulations of a subset of turbines. At least for the NVIDIA GPUs we used, we weren't close to using all of the GPU at 1000 turbines. The GPU solver is also not yet tuned to leverage multiple GPU cores, so there's a lot more potential for speed up yet. I hope this helps and I'm happy to talk more, if you'd like.

[MarcusBecker-GitHub]

Thanks for the comment @rafmudaf!
Do you happen to know if that also holds for the CPU version?
Is FLORIS doing under-the-hood optimization of turbines spaced far apart?
Also, I assume there is no blockage model yet, which we would need to take into account by including a minimum spacing of the smaller wind farms.

[rafmudaf]

The CPU solver scales almost cubic with the number of turbines, so it's much better to split the problem up into smaller farms. There are currently some methods to do multicore processing across the wind conditions (findex), but there are not methods to split the set of turbines for parallel processing. It's a little tricky to do in a general sense since the layout changes with wind direction, but I have some ideas and in the case of multiple distinct wind farms I think it would be easier since the boundaries are known.

Is FLORIS doing under-the-hood optimization of turbines spaced far apart?

No, there's no optimization for that. The CPU solver (aka sequential solver) loops over every turbine in the layout regardless where it is. I think at one time, Paul had a function that was smarter about avoiding the wake calculation for wind turbines that had no impact on other turbines (i.e. the last turbine in row), but I also think that was dropped in the transition from v2 to v3 since the performance improvement from vectorization was substantial.

I assume there is no blockage model yet

That's right, no blockage model yet.

[MarcusBecker-GitHub]

Okay, but that kind of limits this idea - OFF would create $n_T$ windfarms of $n_T$ turbines, leading to $n_T^2$ turbines in one FLORIS simulation. If CPU FLORIS scales with $\approx n^3$, this would lead to $(n_T^2)^3$ with this approach instead of $n_T \cdot n_T^3$ with the current approach.
Then it would make more sense to build a wake solver in OFF that processes the $n_T$ simulations in parallel.
Maybe it does make sense once the GPU FLORIS is ready for use and we build a solver that utilizes it.

[rafmudaf]

How small are the small wind farms? There's a CPU version of the same parallel solver used on the GPU. Since it's computing the entire farm at every iteration, it's a lot of wasted CPU cycles and very slow on large problems. I haven't compared it to the sequential solver for small problems, but it may be practical.

Also fwiw the GPU solver is a pet project and it's not in active development. I'm working to find support for extending it to include additional models (currently only supports Jensen) and tuning it further. In the near term though, I would not plan anything that depends on it.

The process of creating the GPU solver taught me quite a bit about designing a simulation engine like for FLORIS for speed. I think it all comes down to memory management, and the current FLORIS architecture doesn't handle this as well as it could. If you choose the path to implement a wake solver in OFF, I'd like to share what I've learned so that you can avoid the same limitations.

[MarcusBecker-GitHub]

It's not a pressing issue; your post just got me thinking if maybe that's a better way to approach it.
Another idea I had was to look into the FLORIS implementation of the layout optimization and see if we can copy some implementation there for our purposes.
But looking at layout_optimization_scipy.py, it seems to me like you are also using self.fmodel.set(layout_x=layout_x, layout_y=layout_y) followed by self.fmodel.run().

There are now a couple of people interested in OFF / using it, so I think a user/developer meeting would be nice. Maybe I should also have an OFF hackathon ^^ In preparation for that, it would be nice to hear about your learnings.