What you're bothered by is that there is no ultimate truth even in mathematics. In the end you always have the base axiomatization that is not proved but assumed. One axiomatization serving as the basis for math is ZFC set theory. It includes deduction rules and in the end the very low level math always becomes string matching (as in, if I can prove A and B then I can prove A & B, then I can prove C => A & B for any new statement C). Tried to showcase this here

https://medium.com/p/8b5549f1cf9c

And it's amazing that people can say mathematicians just make up stuff, these theories proved important theorems, that are implemented everywhere around you. It's like going into a quantum computer laboratory and saying that what they do is wrong and they should stop. If you fail to understand something as strong as math, it's your loss, not math's.