Payload grasping and transportation with quadcopters is an active research area that has rapidly developed over the last decade. To grasp a payload without human interaction, most state-of-the-art approaches apply robotic arms that are attached to the quadcopter body. However, due to the large weight and power consumption of these aerial manipulators, their agility and flight time are limited. This article proposes a motion control and planning method for transportation with a lightweight, passive manipulator structure that consists of a hook attached to a quadrotor using a 1-DoF revolute joint. To perform payload grasping, transportation, and release, first, time-optimal reference trajectories are designed through specific waypoints to ensure the fast and reliable execution of the tasks. Then, a two-stage motion control approach is developed based on a robust geometric controller for precise and reliable reference tracking and a linear-quadratic payload regulator for rapid setpoint stabilization of the payload swing. Furthermore, mathematical guarantees are provided for closed-loop stability to ensure safe operation. The proposed control architecture and design are evaluated in a high-fidelity physical simulator and also in real flight experiments, using a custom-made quadrotor-hook manipulator platform.