Improvisation can be intimidated and overwhelming. When any note is a possibility, how do you know which to choose? How can you be confident the notes you choose will sound good? There are two good answers to this.
The first answer is to make use of constraints. This may seem counter-intuitive. Improvisation is all about freedom and playing whatever you want, right? Not quite. All capable improvisors know that music typically follows certain rules. The reason some notes sound "good" and others "bad" when someone is improvising is that the listener has certain expectations about the notes they'll hear in a certain musical context. You can play with those expectations and bend the rules - but you need to understand "normal" before you can go beyond it. So the first step to improvisational confidence is to learn the rules for what will reliably sound "good".
The most basic example of this is that if you're improvising over a song in a certain key, you should choose notes from the scale belonging to that key. If you do this, then none of the notes you play will sound "wrong". From there you can build up your knowledge of which scale notes go well with each chord in the progression the song is using, which lets you choose notes that sound not just "okay" but really musically effective. This is a simple and reliable approach to gaining confidence with improvisation, by providing you a "safe zone" in which any note you choose will sound okay. The catch is that it can lead to very uninspired and even robotic improvisation.
Guitarists in particular are prone to getting stuck walking up and down scale patterns without feeling they have much creativity in their solos. To take advantage of this "safe zone" while still being creative you might think the solution is to remove or relax the constraints. In fact the opposite is true! Particularly if you consider yourself "not very creative", adding more constraints to your improvisation is a powerful way to be more innovative and enjoy improvisation more freely. You can constrain the notes you choose from, for example trying to improvise with just the first three notes of the scale. You can constrain the rhythm you play, for example deciding that you'll play a bar of triplets and then a bar of quarter notes. You can constrain your playing technique, for example playing all notes staccato or legato.
More examples of constraints. With each constraint, see how much you can vary your improvisation in other ways while still obeying that constraint. As you do this, you build up a sense of all the different ways you can vary your improvisation, all while staying in a "safe zone". You also get the chance to explore your own taste for what you think sounds good. Employing constraints in this way is an easy and effective way to establish your own creative toolkit for improvisation that you can be confident lets you create interesting, varied, and good-sounding improvisations every time.
The second answer to getting confident and creative with improvisation is ear training. The approach described above lets you play notes without necessarily knowing how they'll sound before you do. You're relying on the systems, rules and constraints to make sure your improvisation sounds musical and interesting.
Arguably a better approach is to equip yourself with the skills to bring out the music you imagine in your mind out into the world through your instrument. This is, in a sense, a "purer" form of improvisation, relying solely on your own inner creativity. People who say they’re “not creative” normally mean they find it hard to create their own music on their instrument. But if you ask them to imagine a killer improvised solo, they actually probably could come up with something in their mind, and it would almost certainly sound pretty musical, based just on our instinctive understanding of what makes music sound like music. So the missing piece is to train your ears for notes, chords, progressions and rhythm, so that you can take that imagined improvisation and directly play the corresponding notes on your instrument.
This takes dedicated practice but has the huge benefit that you can be confident that your improvisations will not only sound good - they'll be 100% what you wanted them to sound like, and unbounded by any constraints except your own imagination.
Christopher Sutton is the founder and director of Musical U, a website and community dedicated to helping musicians develop the "inner skills" of music like playing by ear, singing in tune, improvising and writing their own music. He also hosts the Musicality Podcast interviewing world-leading experts on these topics and more.