Jim
You could try using un-sanded tile grout as mortar. Mix it up as described on the box and spoon it on each wall using a sponge to work it into the gaps. Since it is made for grouting tile it is relatively easy to use like this.
I originally built this pump house by gluing flat stone pieces onto a basic form made from pink insulation board. Once dry I grouted them. The shinny coating is driveway-cement sealer. When I get around to weathering it will not have the sheen.
Like your building the stones are not smooth or flat so the grout gathers in the low spots. I could have removed all of this with more effort. I will put it down to learning curve as this was my first attempt at this. For those who are curios the stones came from a granite dealer and off-cuts from smoothing pieces to make tombstones. They are about 3/8" to 1/2" thick and 1/2" wide. I cut them to link using tile cutter pliers.
Don’t give up Jim. It is going better than you are giving yourself credit for.