Have you backed the foam with anything?

What Darren said. You'll need to glue or apply a backing on that foam.
Just like your scrim foam has, Brushed Tricot "T-Shirt" backing

I was using some inch twill tape. The two seams are just a 1/2 inch apart. I grabbed some other material remnants I had and got it to work better. I am still getting a better raised section on the back side versus the front.

I've used this method which seems to work well on very narrow pleats. Glue 1/2" pad onto bowlining which strengthens the padding and will stop it from stretching. Mark your lines on the vinyl to 5/8" for 1/2" finish allowing the stretch of vinyl to go side to side on the pleat. This allows the narrow pleat vinyl to stretch over the foam when sewn. Glue the vinyl onto 1/8" pad. Lay on top of foam (no glue) and sew.