I was in landscaping for over 20 years and I'm afraid I don't have a happy answer for you from my point of view. and I'm sorry you are going through this. I used to spend a great deal of my days fixing poorly done work.
First of all, if I might ask, how deep was the fill? If it was over 4", 3" minus might not be a bad idea as a base beneath 1" or 3/4 minus. But the 3" should be leveled and compacted before top coating with a 1 1/2 - 2" layer of 3/4 minus. The stone dust or mason's sand should be placed over that and watered in and allowed to dry if possible before setting your stone. From the looks of the picture there is round rock in the fill. Round rock does not compact. It should be actual base rock. And I hate to add that if you are dealing with a substantial fill, it is a good idea to put fabric under it to prevent sinking areas as the dirt beneath will try to push up through the rock. That of course depends a lot on what's beneath the rock you are putting down for your base. Here we deal with a lot of clay and loam so it is pretty much a must if you are dealing with a substantial fill.
In Oregon, you are not allowed to perform landscaping work of over $500 on one job without a contractor's license. You don't have to have a contract but you are required to have a license. A quick call to your contractor's board will get you the info for your state. If he is offering to do a job beyond your state's laws, that is on him - not the consumer.
Personally, I would demand that he remove the fill so that a qualified person could do the job correctly. It may cost you a little more in the front end but you likely won't be lifting your stone to re-level after the rains and foot traffic cause settling. It is far more difficult to correct a poorly done job than to do it correctly out of the gate. I contractor that does flat work should be able to look at it and tell you if it is appropriate as a fill. They'd do that as part of providing you a bid so it shouldn't cost you anything to find out.
The wheel ruts would be unacceptable to me. We always brought plywood to jobs where we had to cross areas we didn't want damaged.
As to the stone, as a contractor, I would generally not allow my customer to buy the stone on my license. If he wants to buy the stone and have it delivered, that's fine. No way would I give him my price on that stone though. That mark up is part of my profit margin and a good contractor should make a profit on anything he has a hand in. I instead generally had the customer meet me at the yard and look at the pallets before they were delivered if they had concerns. Being that you are a contractor, you might get his price anyway and that would be fine as long as you were making all the arrangements to get it to your job site.
So there's my input. Most of the stone work and pavers I did was for Intel and Microsoft stock splitters who had more money than brains or the want to do anything for themselves, so keep in mind that money wasn't as huge of a concern as it is to most of us.
Not sure if this helps.
One other thing. If you don't want him on your property to remove the fill and further mess up your site, you can trespass him as the owner of the property. IF he enters anyway, you can simply have him arrested or removed by the local sheriff.