Background Information of System Design
Colour imaging techniques with CCD cameras can be broadly divided into two domains. The first of these is the Tri-colour separation method and the second is the use of a multi-colour filter matrix integrated directly onto the CCD, followed by colour information recovery in the subsequent video processing hardware. This second method is employed in almost all home video camcorders and gives quite good results at a relatively low cost.
The tri-colour separation technique is the 'classical' method of creating a colour image and has been used since the time of James Clarke-Maxwell, who pioneered it late in the last century by using red, green and blue filters to take three photographs on black and white film. The subsequent recombination of the slides when illuminated with red, green and blue light, gave a convincing (although probably false, due to the extreme blue sensitivity of the film) reproduction of a colour scene.
The tri-colour method is a tried and tested way of creating a high quality colour image and is technically superior to other methods. This has led many amateur CCD users to employ this method of colour image generation to take pictures of astronomical objects and many (eg. Don Parker, Jack Newton, Nik Szymanek and Ian King) have produced some outstanding work. However, there are problems with tri-colour imaging , the main ones being as follows:
Three images of the object must be obtained through red, green and blue filters, hence the total exposure time required is multiplied by as much as 10 times, owing to the poor blue response of many chips.
Following from (1) above: The three images must be physically registered with each other to create the final picture and this can be difficult to perform accurately if the pixels are large. This also leads to difficulty if a moving object is being imaged, as the separations will be impossible to register for all objects in the field. Two examples being cometary imaging and the imaging of a rotating object, such as Jupiter.
The strong infra-red response of most CCDs can distort the passband of the colour filters in use so that the colour rendering of the image becomes questionable. For example, the 'red leak' of many blue filters will cause infra-red light to be recorded as if it were blue in the picture.
Subsequent processing of the image can easily distort the transfer curves of the three colour separations so that they no longer 'track' each other when recombined.
An example of this is where the red frame is much brighter than the blue frame and is probably strongly affected by orange skylight from sodium lamps etc. This red bias must be removed by brightness and contrast manipulations and is unlikely to be accurately compensated for. If the images are then subjected to a strong distortion of the transfer curve, eg. by using a 'logarithmic stretch' to bring out faint detail, it is very likely that the faint detail will have a red or blue bias relative to the brighter objects and the image will look unnatural.Related to (4) above: The colour rendering of astronomical objects is wide open to distortion by the over-enthusiastic processor. The temptation to 'add a little more red' to a nebula because 'it ought to look that way', is very strong and the result may look pretty but be way off the truth! We have become conditioned by the highly saturated colours of Voyager and Hubble pictures etc. and expect to see intense colour wherever we look. Most astronomical objects are far more subtly coloured than these images infer and, although increasing the colour saturation can be useful, it should not be distorted if a realistic result is desired. This form of 'creative imaging' is inherent to all colour techniques, but is particularly easy to generate, deliberately or accidentally, when working with a tri-colour separation. The method to be described shortly is less prone to this and so it helps to preserve a reasonable fidelity.
Software synthesis of colour images from a colour matrix CCD:
A few years ago the standard method of generating a colour TV image from a single pick-up device was to incorporate a Red/Green/Blue 'stripe' filter pattern onto the imager surface so that RGB signals would appear in an alternating sequence as the imager was scanned. This was used in the Sony 'Trinicon' camera tube and on some early CCD chips. The technique was quite successful but reduced the camera sensitivity by a factor of about 4 times and gave rather bad 'aliasing' effects in the colour picture (colourful Moire fringing effects on finely patterned clothing etc.). The pressure to develop a better technique has caused manufacturers to move away from this colour stripe arrangement and the latest CCDs for colour video use a 'secondary colour matrix' filter pattern instead.
In this system, a 'checkerboard' arrangement of Yellow, Magenta, Cyan and Green filters is deposited onto the CCD surface in such a way that colour information can be derived without a serious loss of imager resolution and with a minimum of 'alias' patterning. The sensitivity of such a CCD is about one half of that of a monochrome chip and the detail resolution is almost unaffected.
When the matrix CCD is used in a colour camcorder the electronic circuitry of the camera passes the video signal through various electronic filters and the picture information becomes divided into 'red', 'blue' and 'luminance' components (the green signal can be generated by subtracting the red and blue images from the luminance). 'Luminance' is a term which is used to describe the brightness or intensity of an image without any reference to colour (hue).
If we display the luminance signal on a TV monitor, the result will be a black and white picture indistinguishable from that from a monochrome camera, and so a monochrome TV can view such a colour picture without any problems - obviously important if compatibility is to be maintained. This luminance signal has several very useful properties which have a bearing on our attempts to produce astronomical colour images. The luminance can be thought of as being a synthesis of all the important data in a picture minus the colour information, and so it is possible to perform all our image enhancements on the luminance data alone and then recombine it with the colour data for the final display. This gives us an excellent method of getting the most out of a colour picture without distorting the colour rendering! We can contrast manipulate, unsharp mask, median filter etc. etc. without any worry that the sky will turn red or that Jupiter will have blue belts on a green background!
An important fact that makes this possible is that the human eye and brain combination is very sensitive to fine detail in the luminance signal but is very insensitive to colour detail. The colour data can therefore be of quite low resolution if it is to be recombined with a high definition luminance signal for display and this fact is used to great advantage by all colour TV systems world wide.
To prove this to yourself, just turn down the contrast and turn up the colour on your home TV and watch the picture turn into a blurry shadow of its former self! The video data transmitted by a TV station is almost entirely luminance information with a 'bandwidth' of about 5MHz, the colour data being compressed into a 1MHz band multiplexed into the luminance on a 'subcarrier', and having a spatial resolution of only about one fifth that of the monochrome picture.
Because of this physiological quirk of the human eye and brain, it is unnecessary to generate high definition colour data, and the inherently low colour resolution of the filter matrix is not important.
When a colour matrix CCD is used in a camcorder, the hardware does all the work of decoding the signal and no other processing is needed, however, an astronomical camera is operated by quite different techniques and the colour decoding will need to be performed by software on the computer. Fortunately, we know which pixels are looking through which colour filters (from the manufacturers data sheet) and it is possible to decode the wanted data by performing a series of simultaneous equations on the video data signal levels from adjacent pixels. Such equations can give us values for the red, green and blue content of each matrix group and, due to the cleverly arranged filter pattern, we can also generate a high definition luminance signal by taking the average of adjacent pixels in the horizontal plane. A single 'monochrome' image from the colour CCD can therefore be 'decoded' into red, green and blue colour separations, along with a high resolution 'luminance' image!
Once the colour and luminance data has been synthesized by this mathematical method, it can be stored in two separate disk files and the subsequent processing largely limited to operating on the luminance data, as described above. When the image is to be displayed, the colour and luminance data are recombined and a high definition red, green and blue data stream is created to drive the computer monitor with a good quality colour picture.
The colour balance and saturation are easily checked by imaging a neutral grey subject, such as the moon, as the test object and adjusting the equations to give a grey result. If this set up is then used for all subsequent images - you can be sure that the colour rendering is close to reality and so the uncertainties are removed!
Such images have no registration problems, no excessive exposure times, no need for careful processing to keep the colour tracking and no need for a colour filter wheel assembly! Also, the colour filter matrix applied to the CCD is accurately matched to the CCD spectral response curve, and this guarantees that the result will be unaffected by 'red leak' etc.
These advantages are very significant and I think that amateur colour CCD imaging might benefit considerably from the adoption of this technique. Although I have concentrated on the use of a single exposure with a colour matrix CCD, it is possible to devise software to process a tri-colour image by the luminance-chrominance technique and this could benefit many other camera users who are already using filter wheels. After all - you are quite happy with the colour picture on your home TV and that is generated in this manner!!
Many readers will know that I have a liking for that astronomically unpopular device - the Interline CCD! The reasons for this are many, and now I can add one more, in that they are readily available with a built-in filter matrix as described above!
The Sony range of interline CCD chips includes many with the filter matrix, but the best compromise, in my view, is the ICX027CKA-6. This is a 500 pixel per line 'half inch' colour chip with on-chip microlenses and very low thermal leakage.
One remaining general comment on the display of colour images on a PC computer may be in order. Most VGA colour displays are capable of only 256 simultaneous colours and this is not adequate for a really good quality colour image reproduction.
Higher performance cards (eg Cirrus Logic etc.) are becoming more readily available, but they are not yet widespread and it is an advantage if older cards can be used. Most of the colour image processing programs, such as Photostyler, use colour 'dither' to simulate intermediate hues, but this gives a rather grainy result. To avoid this I have used the technique of assigning a primary colour to groups of three adjacent pixels, and the results are quite good, although the image brightness does suffer.
Using this method, each primary colour can be assigned a value between 0 and 63, so that a total of 262,144 colour hues are available and the rendering becomes close to photographic. This method was used to display the various pictures accompanying this paper in the 1024 x 768 x 256 colour mode on a PC VGA screen using a 'Trident' video card.
Although the reader is entirely free to make use of the ideas and information given in this paper, the mathematical principles employed are the subject of a patent application and potential commercial users should contact:
Terry Platt at 'Starlight Xpress CCD systems',
Briar House, Foxley Green Farm, Ascot Rd.,
Holyport, Berks. SL6 3LA.