Eventually I found the problem. File permissions? No. Path had spaces or was the "extensions" subfolder name too long? No.
The php_curl.dll file has file dependances to libeay32.dll, libssh2.dll and ssleay32.dll which all reside in the parent (php) folder but without an entry to that folder in your PATH Environment Variable entry, the dll fails to load.
Although fixing this is as easy as adding the PATH entry, it does have the problem of tying your path to a specific PHP version which means if you need to switch between PHP versions, you really ought to update your PATH environment variable at the same time.
I'm surprised this hasn't been picked up before as both "Running as a Service" and including the php_curl module must both be very common.
I'm running on Windows Server 2016 if that helps.