Foveon X3 Technology In-Depth
Author: David Pitlyuk
Date Posted: April 10th, 2002
http://www.slcentral.com/c/h/a/2002/4/x3

Introduction

An exciting technology has been developed by a company from Santa Clara, California named Foveon. The technology is called X3 which is the world's first full-color image sensor. Before I get into the technology, lets just get some background information on the company itself.

Foveon was founded in 1997 by Dr. Carver Mead:

Before founding Foveon, Dr. Mead was an author and teacher. He has contributed to over 100 publications on subjects such as solid-state physics, microelectronics, and biophysics. He taught at the California Institute of Technology for over 40 years. He is quite well-known in the tech industry and was actually part of the group that founded Intel.

That just gives you an idea about who is behind the company, buts lets just dive right into the good stuff, X3 Technology.

Mosaic Capture Vs. X3 Capture

As I mentioned earlier, X3 Technology is the world's first full-color image sensor, but what is that? First you'll need to understand what an image sensor is. An image sensor is the chip in a digital camera which captures the light colors in order to create an actual image.

The conventional capturing technology in an image sensor is called mosaic capture. The way that mosaic capture works is by using just one single layer of photo detectors in a mosaic pattern to capture the red, green, and blue light. Take a look at this diagram for a better idea:

You can see here that when light passes through the layer of photo detectors it is done processing because of having only the one layer. This means it can only record one color at a time. This diagram better exemplifies that:

This is a side profile of the image before, you can see only one wavelength is able to pass through.

Notice also that there are more green mosaic sensors then any other color, in fact the split is 50/25/25 which means a typical mosaic sensor captures 50% green light and only 25% of blue and red light.

Right there we can see that there is an issue that will definitely affect color quality. Keep in mind though there are other systems like Fuji's Super CCD which uses a honeycomb architecture of octagonal pixels that are arranged on a diagonal axis. Note though it uses a mosaic filter to detect colors, so it is still limited to only one color per pixel.

So how does X3 capture improve on mosaic capture? To sum it up, it can capture three colors per pixel, but lets go more in-depth with this. Remember how mosaic has a one-layer photo detector? Well the Foveon X3 image sensor uses three separate layers which can absorb different wavelengths of light at different depths with each layer capturing a different color. Take a look at this to better show you what I mean:

Above is the difference between mosaic and X3, notice how the corner part shows two more layers below it, while mosaic capture had none.

Now red, green, and blue wavelengths of light can pass through at each pixel rather then only one. This means that rather than having that 50/25/25 ratio or even a 33/33/34 ratio, you get a 100/100/100 ratio at every pixel location.

But you may ask, how do digital cameras capture the colors that they do now if they can't capture 100% of the RGB colors at every pixel? Basically image sensors that use the mosaic capture system have to rely on interpolation for the colors that it does miss. And making a guess at the colors is complex, which means it takes processing power, the more accurate the image sensor can get, the more processing power it has to use. Since guessing that many colors will probably never be 100% accurate, it obviously leads to color artifacts and loss of image detail.

With X3 technology the pictures come out sharper, the colors are truer, and there are fewer artifacts. Since X3 does not need to interpolate it means it does not need to rely on processing power. This significantly reduces the hardware requirements, simplifies the design, and minimizes any lag time between each shot.

Variable Pixel Size

The technology sounds pretty impressive huh? That's what I thought when I heard about it, but there's even more. X3 technology enables a new capability that will open the door to a entirely new breed of camera (as best stated by Foveon). That new breed is a digital video camera and a digital camera that can shoot high-resolution images integrated into one device. While DV camera's can take still pictures, like my brothers Sony DV cam, which saves the picture onto a Sony memory stick, but the quality isn't that great. That's because in order to accommodate both still and video functions the camera must sacrifice performance in one mode to do the other well.

This is where something called variable pixel size comes in. Foveon explains this best:

Because Foveon X3 image sensors capture full color at every pixel location, those pixels can be grouped together to create larger, full color "super pixels."

So this is how it works, signals from a group of pixels can be read as one pixel. So if you have a block of 4 pixels, they can be grouped together and the camera will read it as one. Another good example from Foveon:

For example, a 2300x1500 image sensor contains more than 3.4 million pixels. But if VPS is used to group those pixels into 4x4 blocks, the image sensor would appear to have 575x375 pixels, each of them 16 times larger than the originals. The size and configuration of a pixel group is variable-2x2, 4x4, 3x5, etc.-and is controlled through sophisticated circuitry integrated into Foveon X3 image sensors.

Here's an image which helps to clarify the idea:

So how does this help allow the image sensor to support both video and still without any compromise in quality? Because if you want full-color images at video rates, just reduce the resolution. Since the sizing of pixels can be done in an instant, the X3 image sensor can capture a high-resolution still photo in the middle of recording a video.

The X3 Image Sensor

The Foveon X3 image sensors are produced using a 0.18 micron CMOS processes in high-volume facilities. They are less expensive to manufacture than CCDs. National Semiconductor manufactures the chip in Portland, Maine, but understand that National Semiconductor had no part in the development aspect, that was all done independently by Foveon.

Aside from the technology behind the image sensor there are some other major benefits, first being power consumption. As well all know battery life is a big issue with a digital camera, especially if it's not rechargeable. Since The X3 does not need the micro circuitry that does all the interpolations, it can extend the battery life. A quick note, not needing the micro circuitry will also save money.

With the technologies that I explained earlier in the article image quality and sharpness can increase drastically...in fact...they say as much as three times. Check out this quote from Brian Halla, CEO of National Semiconductor:

At 1-megapixel X3 produces images that are about as good as those shot by today's 2- to 3-megapixel cameras.

That's a pretty impressive statement, but based on some of the demo shots I've seen, well...let's just say those are very impressive as well to say the least. Interested in seeing what the X3 can do? Check out these 3 images:


1MB JPEG (Note: This JPEG was created from a higher-resolution TIFF source file)


1.3MB JPEG (Note: This JPEG was created from a higher-resolution TIFF source file)


1.3MB JPEG (Note: This JPEG was created from a higher-resolution TIFF source file)

Not only is the quality good, it is said that chip will eventually cost about a fifth of the price of its traditional equivalent.

Speaking about price, how much is it going to cost? I talked to Foveon's press person asking if she knew what the chip would cost to a manufacturer but that information was not yet available. We do know that Sigma is the first company to make a camera using the Foveon X3 image sensor. The camera is called the SD-9 and Sigma will begin taking orders for the digital SLR camera at the PMA show in Orlando on Feb 24, 2002 between $2,500 and $3,000 (the price has not yet been set). The camera should begin shipping in May.

While it is not publicly known yet, it has been said they have signed agreements with a few other larger camera companies to design cameras using the X3 chip.

While the price is a bit expensive, keep in mind the camera is an SLR and is 3.5-megapixels. Note that the image sensor is said to generate 2-3 times the image-producing power. Brian Halla also stated that consumers will be able to buy cameras based on a less powerful X3 by Christmas for $300-$400.

The F7 Image Sensor

Here are the features and specs on the F7 image sensor which uses X3 technology:

Features

Foveon® X3™ Pixels

  • Three photodetectors are layered to achieve full-measured color in each pixel.
  • Images have improved sharpness and immunity to color sampling artifacts such as moiré patterns.
  • In contrast to color filter arrays that use light-absorbing filters, the X3 technology converts light of all colors into useful signal information at every pixel location.

Variable Pixel Size (VPS)

  • Several neighboring pixels can be grouped together on-chip to obtain the effect of a larger pixel.
  • Enables flexible video capture at a variety of resolutions.
  • Enables higher-ISO mode at lower resolutions without sacrificing low-noise performance.

Ultra Low Power

  • Use of the most advanced CMOS process technology allows for ultra low power.
  • Input voltages to the sensor are less than 2.5 V.
  • Power consumption is less than 50mW during readout, less than 10mW in standby mode, and less than 100µW in power down mode.

Low noise

  • Foveon has pioneered the use of CMOS sensors for high-quality image capture.
  • The Foveon X3 image sensor offers extremely low-noise readout and very high dynamic range.
  • Proprietary readout circuits eliminate many of the fixed pattern noise artifacts commonly associated with CMOS image sensors.

Blooming Community

  • The Foveon X3 image sensor has substantially greater resistance to the blooming that is characteristic of CCD image sensors.

Specs

Total Pixels 2304 x1536 x 3 Format of X3 pixels within the array
Effective Pixels 2268 x1512 x 3 Format of optically active X3 pixels
Effective Photo Detectors 10.3 million Total number of measured color data points captured by image sensor
Pixel Pitch 9.12µm Center-to-center spacing of X3 pixels
Effective Area 20.7mm x 13.8mm  
Effective Diagonal 25mm  
Aspect Ratio 3:2  
ISO 100 Full resolution mode
Frame Rate 2FPS for Full Array;
25FPS for 576 x 384 Resolution (VPS)
Maximum number of frames per second in the rolling shutter mode
Variable Pixel Size Increments Powers of 2, independently in each direction Number of X3 pixels of one color to average together for output

The F10 Image Sensor

Here are the features and specs on the F10 image sensor which uses X3 technology:

Features

Foveon® X3™ Pixels

  • Three photodetectors are layered to achieve full-measured color in each pixel.
  • Images have improved sharpness and immunity to sampling artifacts (moiré).

Variable Pixel Size (VPS)

  • Several neighboring pixels can be averaged together to obtain the effect of a larger pixel.
  • Enables flexible video capture at a variety of resolutions.
  • Enables higher ISO mode at lower resolutions without sacrificing noise.

Parallel On-Chip A/D Conversion

  • Three integrated 12-bit A/D converters running at up to 20MHz..
  • Red, green, and blue pixels are multiplexed onto a single 10-bit tri-state output data bus.
  • RGB analog outputs are also available on-board.

Integrated Digital Control

  • Minimal external control logic required.
  • Data bus from the image sensor can be connected directly to DSP or video capture bus.
  • Sensor control is via simple 3 wire serial interface.

Ultra Low Power

  • Advanced CMOS process technology results in ultra low power requirements.
  • Power consumption is less than 400mW during readout, less than 10mW in standby mode, and less than 100 µW in power down mode.

Low noise

  • The Foveon X3 image sensor offers extremely low-noise readout and very high dynamic range.
  • Proprietary readout circuits eliminate many of the fixed pattern noise artifacts associated with CMOS image sensors.

Blooming Community

  • The Foveon X3 image sensor has substantially greater resistance to the blooming that is characteristic of CCD image sensors.

Specs

Total Pixels 1344 x 1024 x 3 Format of X3 pixels within the array
Effective Pixels 1280 x 960 x 3 Format of optically active X3 pixels
Effective Photo Detectors 3.7 million Total number of measured color data points captured by image sensor
Pixel Pitch 5µm Center-to-center spacing of X3 pixels
Effective Area 6.4mm x 4.8mm  
Effective Diagonal 8mm  
Aspect Ratio 4:3  
ISO 100 Full resolution mode
Frame Rate 10FPS for Full Array; >30FPS for 640 x 480 resolution (VPS) Maximum number of frames per second in the rolling shutter mode
Variable Pixel Size Increments Integer Multiples Number of X3 pixels of one color to average
Package CLCC 100 Pin  

Conclusion

Overall this technology is very impressive from the looks of it, not only does image quality improve drastically, power is saved, we can now use digital video and still shots at excellent quality, and it looks as if the price will eventually be lower. At a time where the digital imaging market is booming, Foveon looks like they have come up with the proprietary technology that will be behind next generation digital video and photography. I wouldn't be surprised to see every company use the X3 image sensor in the future, although that's considering other companies don't have anything up their sleeves since I doubt they would just let anybody come by and take the industry by storm. I guess we'll just have to wait and see :)