When designing autonomous vehicles, the system needs to be prepared for a number of different problems. Their software is based on three main abstract layers: perception, planning, and actuation. The planning layer deals with short- and long-term situation forecasting, which is crucial for intelligent vehicles. Whatever method is used for forecasting, the dynamic environment of vehicles must be processed for accurate long-term forecasting. In this article, the focus is on maneuver detection. The aim is to detect a lane change maneuver of a given vehicle in a highway situation by analyzing the dynamic environment around it. For this purpose, the time series of Occupancy Grid images and use Convolutional Neural Networks and LSTM are used for processing. A 3-dimensional Residual Convolutional Encoder and a 2-Dimensional Convolutional Encoder in combination with LSTM are compared. It is found that the 3-dimensional convolutional network is more suitable for the analysis of the time series than the recurrent one, because it has significantly higher F 1 scores in the left or right lane changing and lane keeping maneuver as well.