Sometimes VSCodium search functionality for extensions doesn't work, or doesn't return the desired results. So to fix this I had to look for an alternative:
Download the extension from VSCode marketplace, it'll be a .vsix file, then do like the image below.
Source: https://github.com/microsoft/vscode/issues/108147#issuecomment-704723949