This article proposes an innovative inverse problem (IP) approach to demodulate interference signals from a triaxial optical borehole tiltmeter interrogated by a fiber Fabry-Perot interferometer to 13 nm equivalent precision. The IP technique calculates the desired measurand, here displacement, by determining the five inherent optical parameters of the interferometric system using a minimization process over 3 phases: locking mode, learning mode and running mode, to achieve highly accurate estimation of the real optical parameters. Experimental results demonstrate the IP method capable of demodulating nm-order displacements. It can track fast movements up to 133.8 rad/ms in optical phase evolution, corresponding to 0.46 of the Nyquist limit for our instrument. A relatively low normalized RMS value (< 0.073) achieved during the entire demodulation process is a direct measure of phase tracking performance. Moreover, the processing time required to achieve these results with IP approach is shorter than the data recording time, thus suggesting real time capability of the algorithm. This experimental validation demonstrates the applicability of this IP approach for multiplexed optical interferometry for geophysical tilt as well as for numerous application fields in metrology and geophysics.