Lightbox for Gallery & Image Block

Yes! Just set the Link at the Toolbar to Media File

Set Media File to None or remove the Link.

Since WordPress 5.6 you can now set the default behavior for Link to. Go to yourblog.com/wp-admin/options.php and search for image_default_link_type. Set the value to file and hit save. This will apply to all new Image & Gallery Blocks.

Or you can add the follow snipped (WordPress 5.7+ / PHP 7.4+) to your functions.php:
add_filter( 'option_image_default_link_type', fn () => 'file' );

You can change the CSS selector to a gallery (or galleries) containing <a> tags used by baguetteBox.js with the baguettebox_selector filter:
add_filter( 'baguettebox_selector', function( $selector ) { return $selector . ',.my-gallery'; } )
You can override the full selector by just returing your selector e.g. to show all images in your post in one lightbox (not per Gallery/Image Block):
add_filter( 'baguettebox_selector', function() { return '.entry-content'; } )

If you use a Gallery or Image Block outside a post e.g. inside a Widget and want to apply the Lightbox, you need to ensure that the required baguettebox assets (script & style) are queued using the baguettebox_enqueue_assets filter.

If the Widget is on every page or the majority of pages, you can just enqueue the baguettebox assets everywhere. Just add the following snippet to functions.php:
add_filter( 'baguettebox_enqueue_assets', '__return_true' );
If your Widget is just at the front page, use is_front_page():
add_filter( 'baguettebox_enqueue_assets', function( $enqueue_assets ) { return is_front_page(); } );
If you want to use the Gallery & Image Block Lightbox plugin with a block type that is not supported by default, you can make use of the has_block() function. For instance, in the case of the Kadence Blocks Advanced Gallery, use:
add_filter( 'baguettebox_enqueue_assets', function ( $enqueue_assets ) {
return $enqueue_assets || has_block( 'kadence/advancedgallery' );
} );
Of course, in the previous example you also have to add the appropriate baguettebox selector, i.e.
add_filter( 'baguettebox_selector', function( $selector ) {
return $selector . ',.wp-block-kadence-advancedgallery';
} );

By default, baguetteBox.js shows a slide-in animation on image transitions. You can change it to the following filter to 'fadeIn':
add_filter( 'baguettebox_animation', function () { return 'fadeIn'; } );
Or disable the animation completely with 'false':
add_filter( 'baguettebox_animation', function () { return 'false'; } );

You can remove the link from the image or add the CSS class no-lightbox to the image link. Not to the image directly; you have to add the class to the link.