I also tried pencil on paper from the vintage music manuscript jotter I found. Just as my first attempts with a pen, pencil doesn't really work either. I thought my marks were weird until I found the characters worked better without the comic-strip style shading I normally use. At first it was frustrating, but the advantage of using pencil is the ability to erase, so I kept reworking characters over and over until they began to develop into the vision in my head. I discovered the old manuscript paper is very strong and withstood constant rubbing out, which I know for a fact I can't do with the Japanese paper I normally use.
I should probably use pencil sketching in the time-honoured tradition more often to develop new ideas instead of going directly to pen on paper.