Happily there are exceptions to this rule and Curio is one of them. Thanks go to Adam for pointing me to it. Competitively it's in the same space as OmniGraffle with some notable differences. Where OmniGraffle is quite clearly designed for chart making Curio is designed for much more free-form, creative brainstorming. My perfect app would be the offspring of the two.
I've been corresponding with the developers a bit, mainly requesting features, and they've been great at accepting suggestions and offering pointers on how to use it. My only real complaint about Curio is the price. $99 is pretty steep, all things considered. I can't help but wonder if the developers are setting the price based on the amount of effort it took to create Curio rather than as a reflection of the total value it offers to the user.
That said, my experience with it and with the developers leaves me with little doubt that Curio will become an app worth $99 some day, I just hope it's sooner than later. If you're looking for a brainstorming/notetaking/designing app give Curio a look.