The paper presents a novel observer design method for an autonomous vehicle-oriented estimation problem. The design process combines two approaches: the Linear Parameter Varying framework and the error-based ultra-local model. The main goal of the error-based ultra-local model is to deal with the uncertainties and the nonlinearities of the model, whose effects cannot be taken into account during the modeling process. In this way, the performance of the LPV-based observer can be significantly improved. The proposed method is implemented for the estimation of the lateral velocity. The efficiency and the operation of the observer algorithm are presented through simulations in CarMaker and using real test measurements from ZalaZone proving ground.