Deep reinforcement learning is a promising technique that can help create autonomous agents. However, it is still an open problem how one can create a controller with robust operation for real-world automotive systems. The difficulty lies in either sample efficiency for real-world learning or developing a good enough simulator for training. This paper addresses the latter, proposing a method that provides a solution to the sim-to-real gap through domain randomization, learning with disturbances, and observation preprocessing. The method is validated on a small-scale F1TENTH-type test vehicle, that is trained to race autonomously in a fully end-to-end manner. It is demonstrated that the training process results in a policy that can drive the car safely even over the grip limit.