This is just a short blog entry as I didn't have much time to work on my research. I now have the display working 100%. I got a little impatient and started digging in a little deeper myself. In the end I added spi_bcm2835 into /etc/modules to force SPI to start first and that fixed the issue. I posted my fix on the adafruit github for that issue, so hopefully that will help some other people out.

Another thing I realized that would make some more pins available is that I could just use Bluetooth Headphones. Then I could have my adjustable backlight pin back. I went ahead and ordered a broken TI-83 off eBay for $8 including shipping so that I won’t have the hole that I made for the audio port anymore.