Sounds like the situation I had with my engine.
All my cams journals looked like that. The upper caps on the head were deeply scored and it was due to 2 things.
I even had a rocker seize up and fall out, the cam lobe on the number 1 cylinder exhaust cam was smacking the rocker while it was falling out. One day the engine sounded just like a WRX boxer engine and I didn't know what it was until I pulled the valve cover off, saw the cam lobe and the rocker sitting in the bottom of the head. I couldn't believe it. The engine was supposedly rebuilt by an experienced builder in Salt Lake City. It didn't look like it.
1) Engine was assembled dirty
2) Oil starvation.
Check the oil supply hole on the head where the head bolt runs through and check for depth on it and clearance compared to the head stud.
Link to Jafro. Awesome stuff.
Jafromobile
The head that was on my car was previously line honed and I have been told in the past that is a very poor decision for trying to fix a warped head. It is better to heat cycle a head and straighten it but most people don't do it because of know how or clients don't want to pay for it. So I have been told.
My head was junk and had to rebuild another.
I also had to replace all main and rod bearings. The oil pan was dented and had low clearance for the oil take up tube. Stupid.
More was wrong on my engine that it sounds like on yours I am sure. Sorry for rambling but if I can help save you some useless heart and head aches in the future, so be it.
Now is the time to make sure you have everything that is critical correct. Oil port mods in the head, etc.
The only thing I wish I would have done differently was to leave the balance shafts in the engine. I know that sounds crazy, but I drive the car every day and I miss having a totally smooth engine. My balance shafts were scored on the the journals and didn't pass a fingernail test, so they were tossed and I went with a AMS balance shaft eliminator kit.
Check out some other videos by Jafro, that guy has some good stuff.
Good luck man.