This paper presents a system that is able to automatically identify, segment, and track the cross-section of the internal jugular vein (IJV) and the common carotid artery (CCA) in an ultrasound image feed during a central venous catheter (CVC) placement procedure. The goal is to provide assistance to the practitioner in order to decrease the probability of complications stemming from inadvertent punctures of the CCA during the procedure. In the system, a modified Star algorithm is implemented to segment and track the blood vessel throughout an ultrasound video feed. A novel algorithm based on a cascading classifier is used to identify the location of the IJV and the CCA for two main tasks: (1) selecting the initial seed point at the start of tracking and (2) validating the segmentation results at each subsequent frame. The classifier uses shape features (vessel area, ellipse fitting error, vessel depth, vessel eccentricity) and pixel-based features (pixel intensity and the histogram of oriented gradients descriptor) to differentiate between vessel and non-vessel structures and also differentiate between the IJV and the CCA. Evaluated on a database of 800 ultrasound images containing the cross-section of both vessels, the cascading classifier was able to identify the IJV and the CCA in 92.25% and 85.13% of the images respectively without any initialization from the user at a maximum processing rate of 40.65 frames per second. This allows identification to be conducted in real-time with existing ultrasound machines. © 2017 IEEE.