How do operating systems draw to the screen?
This thread may be quite long as I'm trying to learn about how an OS such as Windows or Linux distros actually draw objects to the screen this is including windows,boxes,cursor,images,web pages,videos etc or in other words how does the OS have a GUI? Hopefully this thread will be a go to when someone on here needs to get a better understanding of how this process actually takes place.
I am one of the many people who believe in the magic of objects appearing on the monitor and not questioning it but I'd like to now know the nuances of what's actually happening under the hood.
My speculation is that there is a very large buffer in memory somewhere that the graphics card(gpu) can write to, the buffer contains all the pixels of the screen, each pixel has 3 colours, red,blue and green(RGB) by varying the intensity of these three base colours it allows us to create a plethora of colours. So my guess is each pixel is 4 bytes in memory, 1 byte for each colour and 1 byte for the transparency/opaqueness of said colour. Obviously the physical pixels are laid out in a matrix on our monitor and I'm sure they have a similar layout in memory where they are in order. So potentially the graphics card can access each individual pixel such as pixels[row][col] = Pixel(255,126,244,255).
If you actually look close enough you will see each individual pixel, but how are some images better quality than others??
I mean if I can look closely enough I can see each individual pixel on my monitor but let's say we have a low resolution image does this mean that the image only uses some of these pixels? I just can't see how only some of these pixels will be turned on.
I mean if we have an 8 bit image, what does this really mean? yes an 8 bit image I'm guessing can have 256 different possible colour values from 0-255 but does this mean that not all of my tiny little individual pixels(on my screen) are used for 8 bit images?
And how does an OS know how to display an 8 bit image on a monitor that has millions upon millions of individual pixels in contrast to a normal image such as a JPEG?
Thanks
I am one of the many people who believe in the magic of objects appearing on the monitor and not questioning it but I'd like to now know the nuances of what's actually happening under the hood.
My speculation is that there is a very large buffer in memory somewhere that the graphics card(gpu) can write to, the buffer contains all the pixels of the screen, each pixel has 3 colours, red,blue and green(RGB) by varying the intensity of these three base colours it allows us to create a plethora of colours. So my guess is each pixel is 4 bytes in memory, 1 byte for each colour and 1 byte for the transparency/opaqueness of said colour. Obviously the physical pixels are laid out in a matrix on our monitor and I'm sure they have a similar layout in memory where they are in order. So potentially the graphics card can access each individual pixel such as pixels[row][col] = Pixel(255,126,244,255).
If you actually look close enough you will see each individual pixel, but how are some images better quality than others??
I mean if I can look closely enough I can see each individual pixel on my monitor but let's say we have a low resolution image does this mean that the image only uses some of these pixels? I just can't see how only some of these pixels will be turned on.
I mean if we have an 8 bit image, what does this really mean? yes an 8 bit image I'm guessing can have 256 different possible colour values from 0-255 but does this mean that not all of my tiny little individual pixels(on my screen) are used for 8 bit images?
And how does an OS know how to display an 8 bit image on a monitor that has millions upon millions of individual pixels in contrast to a normal image such as a JPEG?
Thanks
most gui work is 2-d and is just directly fed to the graphics card. The card has its own memory, though some very cheap cards use a chunk of system ram for it most have it onboard in the chips.
image quality is just dots per inch (dpi) of how it was captured (photos, scans, etc) or created (drawn by algorithm). Image quality is also damaged by some types of compression: jpg specifically damages the image to save space. Dpi is like how you can zoom in on an icon from a program and it is trash when you look closely, because its only 64 or 128 or whatever pixels across, while a photograph may be 3000 across.
8 bit image can mean greyscale (its the same as r=g=b in rgb color). It can also mean the RGB color scale is divided down into a small set of colors (256, this is from 1990/ dos days!) or even smaller (16 bit color was a thing, red green blue purple cyan magenta white black and a few more combos of those). Finally 8 bit can use a best fit palette of colors from the original RGB, and the palette is stored in the image data/format. So 8 bit has multiple meanings. It just means how many bits make up the colors. You can have a 10k by 10k 8 bit image, eg a very high res greyscale image. Your screen pixels are all used for that, and it won't even fit without being reduced in size!
the OS just dumps the raw pixels into a rectangle on the screen, converting from its native format into the local one, eg 8 bit to 16 million RGB conversion then dumped. Again, consider greyscale, it just sets r=g=b for the 3 color pixels and dumps that image to the screen. The os does not know a lot of image formats, though. It works internally for its windows and icons and mouse pointers etc in some format, and for image files you typically need a program to display them, the OS has no clue what to do with them.
You are confusing image processing with the OS and its inherent graphics engine somewhat here. They are distinct topics.
jpeg is NOT normal by the way. it is a numeric transform called a wavelet (google wavelet image compression), specifically using the wavelet known as the discrete cosine transom. To display a jpeg you need a LOT of code to do the math (its fast, but its a fair bit of processing) to get it back into RGB format. There are far better wavelet algorithms; jpeg2000 was one but these formats take even MORE math and the CPUS of the time could not display the images fast enough due to the intense conversions, so these other formats did not gain acceptance when they came out and never took off later even though computers can handle them easily now (this was before multi core processors and all, back in the 300ish MHZ pentium cpu era). I had an experimental wavelet tool that could get about 50% of the jpeg size for the same quality, but it was very slow. A NORMAL image is the RGB or RGBA format in the raw, which is ~30 megabytes per image from a fairly standard camera (3k*3k*3 bytes). A normal video uses 33 or so frames per second of THOSE and is ginormous. Almost nothing deals in raw video stream but you can do it for high quality snippets. MPEG is motion jpeg, the frames are compressed with jpeg and then compressed again by changes since last frame over time, so it is much smaller than this but once again a lot of processing is going on.
image quality is just dots per inch (dpi) of how it was captured (photos, scans, etc) or created (drawn by algorithm). Image quality is also damaged by some types of compression: jpg specifically damages the image to save space. Dpi is like how you can zoom in on an icon from a program and it is trash when you look closely, because its only 64 or 128 or whatever pixels across, while a photograph may be 3000 across.
8 bit image can mean greyscale (its the same as r=g=b in rgb color). It can also mean the RGB color scale is divided down into a small set of colors (256, this is from 1990/ dos days!) or even smaller (16 bit color was a thing, red green blue purple cyan magenta white black and a few more combos of those). Finally 8 bit can use a best fit palette of colors from the original RGB, and the palette is stored in the image data/format. So 8 bit has multiple meanings. It just means how many bits make up the colors. You can have a 10k by 10k 8 bit image, eg a very high res greyscale image. Your screen pixels are all used for that, and it won't even fit without being reduced in size!
the OS just dumps the raw pixels into a rectangle on the screen, converting from its native format into the local one, eg 8 bit to 16 million RGB conversion then dumped. Again, consider greyscale, it just sets r=g=b for the 3 color pixels and dumps that image to the screen. The os does not know a lot of image formats, though. It works internally for its windows and icons and mouse pointers etc in some format, and for image files you typically need a program to display them, the OS has no clue what to do with them.
You are confusing image processing with the OS and its inherent graphics engine somewhat here. They are distinct topics.
jpeg is NOT normal by the way. it is a numeric transform called a wavelet (google wavelet image compression), specifically using the wavelet known as the discrete cosine transom. To display a jpeg you need a LOT of code to do the math (its fast, but its a fair bit of processing) to get it back into RGB format. There are far better wavelet algorithms; jpeg2000 was one but these formats take even MORE math and the CPUS of the time could not display the images fast enough due to the intense conversions, so these other formats did not gain acceptance when they came out and never took off later even though computers can handle them easily now (this was before multi core processors and all, back in the 300ish MHZ pentium cpu era). I had an experimental wavelet tool that could get about 50% of the jpeg size for the same quality, but it was very slow. A NORMAL image is the RGB or RGBA format in the raw, which is ~30 megabytes per image from a fairly standard camera (3k*3k*3 bytes). A normal video uses 33 or so frames per second of THOSE and is ginormous. Almost nothing deals in raw video stream but you can do it for high quality snippets. MPEG is motion jpeg, the frames are compressed with jpeg and then compressed again by changes since last frame over time, so it is much smaller than this but once again a lot of processing is going on.
Last edited on
This could be going back to the dpi concept you talked about, but we have a finite amount of pixels on our monitors, let's say our screen resolution is 1366 x 768 pixels.
and again let's take an 8 bit image, such as this - https://i.pinimg.com/originals/c5/0e/0a/c50e0a205f5d839a09c2239e45dee376.png but much smaller.
As we have a set amount of physical pixels on our screen, how come some images can use less pixels to actually render an image like the one I showed above?
Using the above picture again, the 8 bit image obviously has to have the same amount of pixels turned on as it would with any other item it is rendering to the screen, I am guessing that it uses more pixels to display maybe like a virtual pixel or a grid, maybe 2000 pixels make up one virtual pixel on this grid? (and this could be going back to dpi you mentioned), but if it does this, wouldn't this image still be pretty large because I mean it's still using the same physical screen pixels(1366 x 768 pixels), there's no way of getting around this right?
and again let's take an 8 bit image, such as this - https://i.pinimg.com/originals/c5/0e/0a/c50e0a205f5d839a09c2239e45dee376.png but much smaller.
As we have a set amount of physical pixels on our screen, how come some images can use less pixels to actually render an image like the one I showed above?
Using the above picture again, the 8 bit image obviously has to have the same amount of pixels turned on as it would with any other item it is rendering to the screen, I am guessing that it uses more pixels to display maybe like a virtual pixel or a grid, maybe 2000 pixels make up one virtual pixel on this grid? (and this could be going back to dpi you mentioned), but if it does this, wouldn't this image still be pretty large because I mean it's still using the same physical screen pixels(1366 x 768 pixels), there's no way of getting around this right?
Last edited on
when it converts it to draw it, into RGB or RGBA, it can also resize it up or down. This can be done using a variety of algorithms all lumped under the term 'interpolation'. For example, if you want to quickly make it smaller, you can take the average color of a 3x3 to make 9 pixels into 1, or similarly average 2 rows or 2 columns and so on. There are better approaches, but these work with some distortion.
you can likewise create a row or column or pixel to make it larger.
you have to map the image to the screen, yes. So yea, to display a little image in a big chunk of screen, each pixel becomes many, and at some point that looks bad as the pixels explode into blocks or if its trying to be really smart they may become pixelated circles.
worse, the screen isnt square, so pixels are NOT square on the screen. The graphics card and image processing have to deal with that, too.
I mean, just LOOK at the image you gave. each square solid block of color was probably 1 pixel but on screen, it is 50, 100 pixels across the square! That image, by the way, maybe once it was 16x16 but now has been expanded to be much larger. you can try to resize it in an image program to get it back to 16x16 or whatever but the program is likely to blur it up or distort it due to the gridlines etc. It almost looks like a screenshot of it zoomed in in a program.
Or think about what happens when you zoom in on a picture. You can blow a 3x3 pixel spot up to a 1000x1000 onscreen pixels. Its just doing interpolation and resizing it then drawing to screen.
you can likewise create a row or column or pixel to make it larger.
you have to map the image to the screen, yes. So yea, to display a little image in a big chunk of screen, each pixel becomes many, and at some point that looks bad as the pixels explode into blocks or if its trying to be really smart they may become pixelated circles.
worse, the screen isnt square, so pixels are NOT square on the screen. The graphics card and image processing have to deal with that, too.
I mean, just LOOK at the image you gave. each square solid block of color was probably 1 pixel but on screen, it is 50, 100 pixels across the square! That image, by the way, maybe once it was 16x16 but now has been expanded to be much larger. you can try to resize it in an image program to get it back to 16x16 or whatever but the program is likely to blur it up or distort it due to the gridlines etc. It almost looks like a screenshot of it zoomed in in a program.
Or think about what happens when you zoom in on a picture. You can blow a 3x3 pixel spot up to a 1000x1000 onscreen pixels. Its just doing interpolation and resizing it then drawing to screen.
Last edited on
Still kind of confused as to how different images can be of different sizes when they are using the same psychical pixels, for example how can a grey scale image be smaller than an RGB image, I mean as you mentioned the same pixels have to be drawn/mapped by the graphics card to the screen, so how is it that the colors to represent a grey pixel will take up less memory than an image that uses color?
And that brings me on to my next question about color depth and it's kind of the same premise of the question above, how can images have greater color depths when the same amount of pixels are used?
Sorry If I'm not following so far.
And that brings me on to my next question about color depth and it's kind of the same premise of the question above, how can images have greater color depths when the same amount of pixels are used?
Sorry If I'm not following so far.
Last edited on
well, a grey image can be stored as 1 byte per pixel, and RGB is 3 bytes per pixel, so its 1/3 the physical disk size, when uncompressed.
Most likely when drawn to the screen they are the same size. The conversion to screen data is probably sending the grey in RGB format after a conversion. Not 100% sure, maybe this is avoided somehow, but that is what I believe happens. The memory used when drawn and the file size on disk can be radically different; same for jpeg which uncompressed to 10 times or so its disk size when drawn.
the second question is nonsense.
you are confusing the number of pixels (the dimensions of the image) and the number of colors each pixel can have.
think of it this way:
struct rgb
{
unsigned char r; unsigned char g; unsigned char b;
};
rgb picture[1024*768];
the size of the array, the image dimensions are 1024*768 pixels.
each pixel can have one of over 16 million combinations (3 bytes, 24 bits, 2^24 combinations of bits).
an image with a single pixel can still have any of 16 million colors.
on the other side, an old 8 bit image from the dos era could have thousands of pixels but each one can only have 1 of 256 values.
Most likely when drawn to the screen they are the same size. The conversion to screen data is probably sending the grey in RGB format after a conversion. Not 100% sure, maybe this is avoided somehow, but that is what I believe happens. The memory used when drawn and the file size on disk can be radically different; same for jpeg which uncompressed to 10 times or so its disk size when drawn.
the second question is nonsense.
you are confusing the number of pixels (the dimensions of the image) and the number of colors each pixel can have.
think of it this way:
struct rgb
{
unsigned char r; unsigned char g; unsigned char b;
};
rgb picture[1024*768];
the size of the array, the image dimensions are 1024*768 pixels.
each pixel can have one of over 16 million combinations (3 bytes, 24 bits, 2^24 combinations of bits).
an image with a single pixel can still have any of 16 million colors.
on the other side, an old 8 bit image from the dos era could have thousands of pixels but each one can only have 1 of 256 values.
Last edited on
I think I could be actually confusing what a pixel is,
when you look closely enough at your computer screen you can (kind of) make out each individual physical pixel, so is a pixel in this context the physical little dot with 3 tiny LED lights (one red,one green,one blue) or is a pixel a combination of these?
The thing I don't understand is how some images (higher def images) can be larger than others(smaller def images)
I'll use an example this image properties is 259 × 194 pixels - https://www.pexels.com/photo/red-dahlia-flower-60597/
Firstly are these pixels referring to the individual little dots on my screen(with 3 LED lights) , so 259 * 194 of these little units? or are these pixels more like a virtual pixel where many of the little units could make up one pixel?
Second question, looking at that photograph, let's say we wanted to make it smaller, we could use a program to reduce the amount of colors it uses per pixel thus making our image smaller BUT what I don't get is we are still using the same little units(3 tiny LED lights) to display our image, by reducing the amount of bits that hold each RGB color are we basically limiting or constraining the intensity of those little lights?
I may need to expand on the above, each of our physical pixels have 3 tiny LED lights, lets take an image with 1 byte for each color(R G and B) so we can have a data type that stores 3 bytes, we can intensify the brightness of each little LED light by increasing it's value, we can choose a value from 0-255 and depending on the value the light will emit a different amount of brightness.
Now lets say we have the same image but let's say it has less variants of colors, so we will have 4 bits for each RGB, so we will have 2^4 or 16 different colors to work with, does this mean now the small little LED lights can only go from a value of 0-15 ( a reduced amount of brightness)? In other words by using a smaller amount of bits for an image are we reducing the power of them little individual LED lights?
when you look closely enough at your computer screen you can (kind of) make out each individual physical pixel, so is a pixel in this context the physical little dot with 3 tiny LED lights (one red,one green,one blue) or is a pixel a combination of these?
The thing I don't understand is how some images (higher def images) can be larger than others(smaller def images)
I'll use an example this image properties is 259 × 194 pixels - https://www.pexels.com/photo/red-dahlia-flower-60597/
Firstly are these pixels referring to the individual little dots on my screen(with 3 LED lights) , so 259 * 194 of these little units? or are these pixels more like a virtual pixel where many of the little units could make up one pixel?
Second question, looking at that photograph, let's say we wanted to make it smaller, we could use a program to reduce the amount of colors it uses per pixel thus making our image smaller BUT what I don't get is we are still using the same little units(3 tiny LED lights) to display our image, by reducing the amount of bits that hold each RGB color are we basically limiting or constraining the intensity of those little lights?
I may need to expand on the above, each of our physical pixels have 3 tiny LED lights, lets take an image with 1 byte for each color(R G and B) so we can have a data type that stores 3 bytes, we can intensify the brightness of each little LED light by increasing it's value, we can choose a value from 0-255 and depending on the value the light will emit a different amount of brightness.
Now lets say we have the same image but let's say it has less variants of colors, so we will have 4 bits for each RGB, so we will have 2^4 or 16 different colors to work with, does this mean now the small little LED lights can only go from a value of 0-15 ( a reduced amount of brightness)? In other words by using a smaller amount of bits for an image are we reducing the power of them little individual LED lights?
A pixel is a unit of data; it isn't the actual physical LED (or whatever makes up your monitor screen). It stores the RGB codes for the correct color to display for a specific LED in the physical screen.
Thus, the fewer pixels, the less information is going to your screen, and it lowers the resolution of the graphic, making it appear "pixelated." And vice versa.
If there are fewer pixels, the graphic will be physically smaller. And if you try to magnify it, that is what results in the "pixelated" display of a low-res image.
Here is a bit of HTML code that may help my explanation:
Save it as a .html file and open it in your web browser; if that doesn't work, I have a couple other things you can try.
Hope this helps!
max
Thus, the fewer pixels, the less information is going to your screen, and it lowers the resolution of the graphic, making it appear "pixelated." And vice versa.
If there are fewer pixels, the graphic will be physically smaller. And if you try to magnify it, that is what results in the "pixelated" display of a low-res image.
Here is a bit of HTML code that may help my explanation:
|
|
Save it as a .html file and open it in your web browser; if that doesn't work, I have a couple other things you can try.
Hope this helps!
max
Last edited on
The example worked great, below is what I'm really struggling to understand
This is exactly what I'm trying to understand, but just doesn't make sense, wonder if you(or anyone) could expand on this, all those little LED lights will have to have a value no matter what so I don't understand how lower res and higher res photos will differ as these each led light has to be some value.
Thanks :)
A pixel is a unit of data; it isn't the actual physical LED (or whatever makes up your monitor screen). It stores the RGB codes for the correct color to display for a specific LED in the physical screen. |
This is exactly what I'm trying to understand, but just doesn't make sense, wonder if you(or anyone) could expand on this, all those little LED lights will have to have a value no matter what so I don't understand how lower res and higher res photos will differ as these each led light has to be some value.
Thanks :)
The screen has pixels. The image has pixels.
Drawing one image pixel with one screen pixel is nice.
Drawing one image pixel with many screen pixels creates "blocky" result.
One screen pixel can show only one value; if the image has more, then you need to combine/average and will lose details.
Take a 1920x1080 image. It shows on 1920x1080 screen full screen.
On 1280x720 monitor you can look only a part of that image, or shrink to see everything at once, but you will lose more than half of the data.
On 3840x2160 screen the image takes only 1/4 of the screen. If you stretch it to full screen, then every pixel of the image has to be shown with 4 (2x2) LEDs.
Drawing one image pixel with one screen pixel is nice.
Drawing one image pixel with many screen pixels creates "blocky" result.
One screen pixel can show only one value; if the image has more, then you need to combine/average and will lose details.
Take a 1920x1080 image. It shows on 1920x1080 screen full screen.
On 1280x720 monitor you can look only a part of that image, or shrink to see everything at once, but you will lose more than half of the data.
On 3840x2160 screen the image takes only 1/4 of the screen. If you stretch it to full screen, then every pixel of the image has to be shown with 4 (2x2) LEDs.
A pixel is a unit of data; it isn't the actual physical LED (or whatever makes up your monitor screen). It stores the RGB codes for the correct color to display for a specific LED in the physical screen. |
That makes sense :) , but how will doing this actually make an image smaller in size(data wise)?
In other words say we make an image more blocky (use more pixels to represent one block(block pixel) ) how does this make a file smaller data wise as opposed to an image that is less blocky but has the same dimensions(uses more screen pixels), considering the screen pixels have to be set to a certain color in both cases regardless, right?
| On 3840x2160 screen the image takes only 1/4 of the screen. If you stretch it to full screen, then every pixel of the image has to be shown with 4 (2x2) LEDs. |
makes sense :)
| A pixel is a unit of data; it isn't the actual physical LED |
| I don't understand how lower res and higher res photos will differ as these each led light has to be some value. |
If you record a sound at 100,000 samples per second (100,000 times per second you measure the relative physical displacement of a membrane and record that measurement) you're going to produce 100 times more information than if you record the same sound at 1,000 samples per second. You're recording the sound at a resolution 100 times higher.
Likewise, if you take a photograph using a digital camera with a 10000x10000 sensor (the sensor consist of an array of 10000x10000 light sensitive pixels), you're recording 100 times more information than if you took the same photograph from the same position using the same camera, but equipped with a 1000x1000 sensor.
If you wanted to display both photos using a standard 1920x1080 monitor, the 1000x1000 photo will fit, but the 10000x10000 one won't, even though they both recorded the exact same real phenomenon. If you want to display the 10000x10000 photo in that monitor at the same apparent size as the 1000x1000 photo, you will need to discard 99% of the information (i.e. lose the resolution). If you want to display it at the same apparent size without discarding the information you will need a 19200x10800 monitor of the same size as the first monitor. In other words, a monitor with 100 times the resolution.
And, the hardware pixels on the screen can be remapped other ways; note that you can 'change the resolution' of your monitor settings in your OS. Clearly this does not alter the hardware: its just remapping the hard pixels in a new way, addressing them differently.
most of this is done by the graphics card, not the OS. The os tells the card what mode to go into, and it flips internal states to produce the desired mapping if possible (if not possible, you get an error or a corrupt display, and modern OS time out if you don't hit ok when this happens so you can go back to a working state easily).
note that changing the resolution changes the sizes of things: if you set it to 640x480, everything is HUGE and many things won't fit properly on the screen. The higher you set it, the smaller stuff gets. But its just that mapping... 640x480 tells the graphics card to draw the stuff such that 1 virtual pixel is 3x3 or whatever it is hardware pixels.
most of this is done by the graphics card, not the OS. The os tells the card what mode to go into, and it flips internal states to produce the desired mapping if possible (if not possible, you get an error or a corrupt display, and modern OS time out if you don't hit ok when this happens so you can go back to a working state easily).
note that changing the resolution changes the sizes of things: if you set it to 640x480, everything is HUGE and many things won't fit properly on the screen. The higher you set it, the smaller stuff gets. But its just that mapping... 640x480 tells the graphics card to draw the stuff such that 1 virtual pixel is 3x3 or whatever it is hardware pixels.
Last edited on
In other words say we make an image more blocky...how does this make a file smaller data wise as opposed to an image that is less blocky but has the same dimensions... |
Through this thread I see several subjects mashing together, though here in these last few posts that has begun to focus on a question like this one.
The answers appear to be correct. You're missing details and not “getting” the answers because the subject requires at least 3 or 4 books, but that knowledge is being compressed, chopped up, and expressed in a combined description which leaves you puzzled, and I sense not quite sure what questions to ask or how they should be framed.
To many of us, after years of study and experiment, all of it seems like one subject, but in reality a student must follow the path from one end to the other, not in a combined salad of everything.
Color theory, which has been part of the discussion (RGB in this case), relates to the construction of the human eye and it's relationship to the light spectrum, and from there winds through a number of realizations that have nothing to do with image resolution.
Image resolution is actually a bit simpler, but you've asked questions about "data wise" relationships to image dimensions. The relationship between the size of data storage required for an image involves, among other things, the color representation used to describe the image. These points have been included in the answers, but I then see you ask the questions again because the subject is vast, you're at a beginning level toward this study and these posts are short by format limitation.
Some of us approach the subject after having been through algebra, trigonometry, linear algebra, quaternion algebra (it was a kind of "lost" algebra before the 70's), and likely some of the Calculus. The language of graphics is the language of math and science.
The subject also involves some of the physics of light, especially for color theory.
Whenever a computer touches reality, the means by which science represents reality through math is involved.
I sense it becomes important for us answering these questions to insist on separating the subject components.
I’ll treat color theory, as that has been lacking so far, and treat images separately in another post.
Color theory applies to each pixel of an image. This is similar, though not identical in all respects, whether we discuss a physical pixel on a display monitor, or a more theoretical pixel in an image not yet on display. It begins, however, outside the computer, with the human eye. We are, after all, modeling human image perception in an attempt to appropriately stimulate the nerve sensors in the retina.
The color of light is due to the frequency if its vibration. The color spectrum of visible light, which you can find through Google, is in the order of the rainbow without coincidence, to be found in the physical use of a prism or the atmosphere full of cloudy mist. Red light is the lowest frequency in the visible spectrum, corresponding to musical notes from a bass guitar. As the frequency increases slightly, the color shifts toward yellow, then green, then blue. Violet, at the highest end we can see, is like the sound of cymbals. These colors can be represented by single frequencies (usually described by wavelength), corresponding to a single musical note by one instrument, perhaps one key on a piano. However, human eyes are only able to sense red, green and blue colors. We have no receptors for yellow or violet. We are blind to frequencies above and below these limits just as we are deaf to extremely low and high sound frequencies.
Curiously, there is no pure color in the spectrum for white. White isn’t a light color. It is a human perception of multiple colors at once, much like a chord in music where multiple notes are combined.
As important as white is in human perception, the fact it doesn’t exist in the color spectrum of light is important for how we model color in light. We see white when all 3 sensors at a retinal pixel location (yes, the eye has these pixels) are equally excited at once. It is when one or two of these sensors receives less stimulation than the others that light begins to take on color in our perception.
This is the reason color is modeled as RGB pixels in a computer. The monitor must stimulate these three receptors simultaneously to full intensity to produce the perception of white in an image, while all the other colors involves only one (red, green or blue) sensor in the eye, or a carefully balanced blend of two or three of these sensors at any one pixel location. The human retina is able to detect changes of light intensity to about 1% gradients (some of us can sense 0.5% at most).
Yellow is an interesting color to explore in this reality. First, there is a pure yellow color in the light spectrum, represented by one single frequency (over a range of a few frequencies). How do we perceive yellow, then, when we have no receptor for yellow in our eyes? How do we see yellow in an image from a device which, itself, does not emit yellow light?
First, consider how the light spectrum gradually shifts through the colors of the rainbow as the frequency of the light increases. Every position between two colors has a frequency, and there is an infinite potential number of frequencies between any two colors.
Each sensor in the eye is sensitive to a particular frequency, but it is not a sharp cutoff. The red sensor, in reality, is most sensitive to red light, but still responds with reduced sensitivity as the frequency increases toward yellow. However, by the point at which the frequency approaches green, the red sensor’s sensitivity is reduced to nearly zero. Similarly, a pure red light is not sensed by a green retinal sensor, but as the frequency increases toward green, the green sensor does begin to respond at reduced sensitivity, reaching it’s most sensitive response at the green light frequency.
So, when we perceive yellow it is because both the red and green sensors respond partially to the yellow light.
What is curious is that if we look at a light source that is pure yellow, and we compare that to a light source which has no yellow at all, but has two frequencies of reduced intensity at red and green, we perceive the exact same color. We are blind to the fact these are two completely different light spectra.
This is why a device which can’t emit yellow can produce the sense of yellow color, because it mathematically emulates the function of the retinal sensors in the human eye.
Violet (or purple hues in darker shades) is another curious version of this. It so happens that violet light is beyond our blue sensor’s range. The blue sensors respond in reduced sensitivity to violet, but not nearly as sensitively as with blue light (which is a lower frequency than violet). Why do we perceive violet? It so happens that the frequency of violet light happens to hit a rather coincidental doubling of the frequency of red light. Our red sensors can pick up a faint hint of the “harmonic” frequency of violet light. If it didn’t, our perception of violet would merely be that it is a less intense blue. Our RGB math modeling of light color, therefore, adds a touch of red to a blue light to emulate this dual sensor stimulation, even though the monitor device itself can’t really emit light at those higher frequencies.
I cover this because without this acknowledgment, the theory behind RGB color values seems rather arbitrary. One should recognize that what we are really doing is mathematically describing the measurement of a physical reaction in the human eye, at each pixel, to real imagery from the real world.
RBG is not the only model, though. It works when the device being controlled emits light, and we’re controlling that light to stimulate the human eye. Ink on paper, however, has a completely inverted model. There, we must mathematically emulate a reverse process, one of light absorption as much as light reflection (a version of light emission). Ink is deposited to absorb the colors we want to hide, so only the color of light we need for the human eye is reflected. For this reason ink uses the CYMK model (Cyan, Yellow, Magenta, Black). I’ll leave the rest to your research, but leave this one point in mind: there is a simple formula for translating between RBG and CYMK (back and forth).
There is another common color model, which is more of a physics reality than of human perception or ink related to human perception. It is the HSL model (Hue, Saturation, Lightness – sometimes said brightness). HSL starts with Hue, a numeric value indicating where on the visible light spectrum the color we’re representing can be found, effectively describing a light frequency. Saturation refers to the depth of this color contribution. If saturation is zero, the hue has no contribution. If the saturation is 50%, that hue is mixed at 50% of full intensity, where the rest is a background of white. 100% saturation suggests no white light, and it is a pure, single frequency of light from the spectrum. Lightness indicates the overall level of intensity. I’ll leave the study of HSL to your own research. It is sufficient to recognize that these are different mathematical perspectives for saying the same thing, though HSL can represent colors way beyond human perception, and can therefore apply to more scientific work beyond visible light. RGB and CYMK are tightly bound to human perception, and can only specify colors humans can see. HSL, when applied to typical imagery, “normalizes” the range of hue to the visible light spectrum, but nothing prevents us from arbitrarily expanding that to frequencies below red and beyond violet, so as to record imagery from astronomical observations from objects emitting light we can never see. For this reason, understanding HSL is more complex, and can move one a little deeper toward physics. The encoding of HSL can be more complex, too, because hue is attempting to communicate a frequency of light, while there would be, in theory, an infinite number of frequencies between the extremes of visible light.
The summary point here is that for any given pixel, we attempt to describe its color, acknowledging that doing so involves the human retina’s function as it responds to the three colors it is designed to perceive. All of this applies to light, or its reflection from paper or real world objects that we view, but says little about each pixel in an image. That is a separate subject for another post.
Topic archived. No new replies allowed.