How do libraries like SDL work?

[adam2016]

Hey guys

So I've always been wondering how libraries like SDL,SMFL,FLTK etc, access the pixels of a computer screen?

from what I have read every pixel is mapped to a frame buffer or a large buffer of memory which as said contains the color of every pixel on the screens resolution.

The CPU will communicate with the GPU and the GPU will render what's in the frame buffer and display it on the users screen ( I could be wrong with this part, please correct If so)

lets say we use a library like SDL or SMFL we can create a window and access the raw pixels, but if I was trying to access these pixels directly that would be impossible ( well certainly improbable ) as the operating system will not allow us to access that portion of memory, yet some how libraries like SDL and SMFL can?

Am I missing something?

thanks

[kbw]

Am I missing something?

Yes.

Software libraries aren't bound directly to hardware. They are bound to some abstract interface defined in software.

This introduces a layer of indirection; indirection from the software to the physical device. This is required to support different kinds of devices that perform the same function.

The software that talks to the physical device is called a device driver. It's the operating system that provides that software interface (API) to the application and an interface for the device driver (SPI) and maps the function calls in the application to hardware instructions on the device. This also allows software emulation of devices.

An alternative arrangement in DOS all those years ago was to write software that talks to a device from one popular vendor. Then all the other vendors would make their devices support that interface. That's what happened with IBM PC text, EGA and VGA for display, Sound Blaster for audio, Hayes for modems, and so on.

Once there was a thriving market for different kinds of hardware devices, MS' next OS, Windows, supported the devices so that application software didn't have to.

This wasn't a new idea, just a new idea for micro-computers. Unix faced the same problem two decades earlier and used this device driver solution to support different physical devices.

lets say we use a library like SDL or SMFL we can create a window and access the raw pixels, but if I was trying to access these pixels directly that would be impossible ( well certainly improbable ) as the operating system will not allow us to access that portion of memory, yet some how libraries like SDL and SMFL can?

Running all these software abstractions takes time. It was a real problem back in the DOS/WIN16 days as it was very noticeable. MS addressed this problem by supporting what was initially called the Game API, later called Direct X, probably called something else now ... It allowed the OS to support applications (games) to bypass all that generic abstraction stuff and go straight to the hardware thru a new API.

[Niccolo]

@Adam2016,

First, the underlying details for SDL and SFML are quite different from FLTK.

FLTK is a GUI library, while SDL and SFML work with the rendering cycle of one of the primary API's like DirectX or OpenGL (or, with some newer libraries, Metal or Vulkan).

The bottom line there is that in FLTK, or other GUI frameworks, there are functions which request bits from the display using OS/GUI API functions on the native operating system. In other words, while you are correct that the OS controls the display, it also answer to display interrogation.

However, typical GUI applications work in the reverse (unless you're taking a screen shot). Typically, applications use a display buffer in the form of a compatible bitmap format, which is then attached to the GUI's drawing API - so the software draws on the bitmap. When complete, the bitmap is blitted to the display as required. This means the application generally has a bitmap of what is on the display (for custom displays) when this is required. For simple dialogs, of course, this isn't the case.

For DirectX/OpenGL (and others), there are ways to render output to a buffer the application can read. There, again, the operating system and the display drivers provide a means of supporting this, so you're not gaining direct access to the display hardware in the application, but receiving a result buffer when requested.

[adam2016]

great insight guys,

so with GUI's lets say we create a window and buttons. On that window with a GUI library such as FLTK or WXWidgets the actual window and buttons are bitmaps like JPEG,PNG etc?

or are the components themselfs of the window (textboxes,radiobuttions, sub images, buttons, etc ) bitmaps that are drawn onto lets say a window ( in windows drawing windows as far as I know is done with help of user32.dll)?

so for libraries like SDL and FLTK which use Diretx and OpenGL unlike GUI libraries like FLTK,WXWidgets they access the raw pixels essentially at a lower level? and would this mean that GUI libraries could be written using applications like SDL?

For DirectX/OpenGL (and others), there are ways to render output to a buffer the application can read. There, again, the operating system and the display drivers provide a means of supporting this, so you're not gaining direct access to the display hardware in the application, but receiving a result buffer when requested.

oh ok so what you get from the OS,is a copy of the pixels but you can't directly write to them instead the operating system will take what you have specified lets say in your program somewhere and it will pass it to the GPU which will in turn display certain pixels on the screen?

The software that talks to the physical device is called a device driver. It's the operating system that provides that software interface (API) to the application and an interface for the device driver (SPI) and maps the function calls in the application to hardware instructions on the device. This also allows software emulation of devices.

that makes sense :)

[malibor]

so with GUI's lets say we create a window and buttons. On that window with a GUI library such as FLTK or WXWidgets the actual window and buttons are bitmaps like JPEG,PNG etc?

or are the components themselfs of the window (textboxes,radiobuttions, sub images, buttons, etc ) bitmaps that are drawn onto lets say a window

Windows, buttons and controls are not bitmaps them self, but just a rectangular area which defines bounds where pixels are drawn for given window or control, these controls or windows provide a drawing context for drawing API's. otherwise drawing API's would draw all over the screen (sort of).

so for libraries like SDL and FLTK which use Diretx and OpenGL unlike GUI libraries like FLTK,WXWidgets they access the raw pixels essentially at a lower level? and would this mean that GUI libraries could be written using applications like SDL?

Graphics libraries such as SDL could be used to draw a window or controls but that would be to hard to implement(for example you would need to implement mouse click and button style when when clicked, and so for everything you potentially draw).

most graphics libraries implement UI interface sub library which is used to create GUI such as windows and controls, these are often tweaked to fit into the graphics engine context for drawing 2D/3D models and effects.

so what you get from the OS,is a copy of the pixels but you can't directly write to them instead the operating system will take what you have specified lets say in your program somewhere and it will pass it to the GPU which will in turn display certain pixels on the screen?

Operating system provides high level drawing API's, these API's are implemented to "talk" to hardware directly such as GPU, going lower than what OS provides makes no sense unless you are driver developer or you're implementing your own drawing API's which most likely means you'll need to write driver code.

And why would need to write to OS's pixels anyway? these are just structures/raw data defined by OS for use by it's UI or drawing API's. I'm not 100% sure but I think these pixels or raw data is precompiled into some sort of special files for to avoid expensive calculations.

[jonnin]

a lot of stuff is flood-fill. Like an edit box background may be a flood fill to the color white, not a image file.

I think you can still draw pixels anywhere in windows, even on top of other programs. It will clear the drawing if you activate the other program, though. Unless they have changed this behavior. This isnt pure direct, you still call something, windows graphics or directx or something.

In windows specifically, directdraw is very, very close to directly drawing on the hardware.