Worth doing for a website, with one caveat
For images served by a web page, yes: WebP is usually meaningfully smaller than JPG at the same measured quality (Google reports 25 to 34 percent). The caveat is that you are compressing an already compressed file. If you still have the original photograph, converting that directly gives a better picture at the same size than going through the JPG.
Some of the saving is just more loss
When a 400 KB JPG becomes a 260 KB WebP, it is tempting to read that as free. It is not entirely free. Part of the difference is WebP being a better codec, and part of it is a second round of approximation removing detail the JPG had kept.
This matters in two situations and in no others. If the image will be enlarged, cropped into, or printed later, start from the original. If it is a hero image or a photo gallery where quality is the point, start from the original. For an ordinary page image at ordinary size, going JPG to WebP is perfectly fine and you should not lose sleep over it.
What you should not do is run the file through repeatedly, or re-encode an image that has already been through three tools. Each pass takes a little more, and none of it comes back.
When WebP comes out bigger
It is not universal. A JPG that was already compressed hard can produce a WebP that is larger than the file you started with, because the WebP is saved at a higher quality setting than the JPG was. WebP wins on average, not always. One more limit: Safari cannot save WebP from a web page, so in Safari the converter says so instead of saving a file; use Chrome, Edge or Firefox for this one.
The tool prints both sizes for every file. When the new one is bigger, keep the original — there is nothing to gain. To see where the real boundary sits for the kind of pictures you work with, the format benchmark encodes your own files at every quality setting and shows you the point where the file starts growing faster than the picture improves.
If your images started somewhere else
Photos straight off an iPhone are HEIC, and converting those to WebP directly avoids a pointless trip through JPG — HEIC to WebP does it in one step. Artwork and screenshots are usually PNG, which converts to WebP with the transparency kept. And if you are going the other way because a WebP will not open, WebP to JPG is the page you want.