X11 does not start on Le Potato, based on base image

I installed Ubuntu desktop image and is working fine, but it is very bloated, so I am trying to build a system from ground up using base image.
I installed X11 and Mate desktop but when I boot the system, I am getting black screen with blinking cursor.
X shows error in the log:
(EE) Cannot run in framebuffer mode. Please specify busIDs for all framebuffer devices

I suspect that I dont have some video driver installed or configured but I dont know how to proceed.
Please help.
Thank you

You need to setup xorg configs.