Library contains all the visual elements you can use in the content. Prefab means a prefabricated object, which are set up in the Object Editor. So essentially, every object that is seen on screen in a tutorial or other content, is an SO (smart object).
Open/Close Library panel
Available Objects are in an alphabetical order in the scrollable column. Hovering and Object shows its name. Object can be added to the scene by dragging & dropping.
You will find insturctions on how to add Smart Objects to your project in the "Object Library" section of this guide.