What an unprocessed photo looks like
No synthesized answer yet. Check the discussion below.
But anyway, I enjoyed the article.
This seems more a limitation of monitors. If you had very large bit depth, couldn't you just display images in linear light without gamma correction.
More importantly, the camera isn't recording blinding brightness in the first place! It'll say those pixels are pure white, which is probably a few hundred or thousand nits depending on shutter settings.
If you kept it linear all the way to the output pixels, it would look fine. You only have to go nonlinear because the screen expects nonlinear data. The screen expects this because it saves a few bits, which is nice but far from necessary.
To put it another way, it appears so dark because it isn't being "displayed directly".
Why exactly? My understanding is that gamma correction is effectively a bit optimization scheme to allocate bits in a perceptually uniform way across the dynamic range. But if you just have enough bits to work with and are not concerned with file sizes, then this shouldn't matter? IIRC LCDs basically have linear response in terms of hardware anyway, and just emulate the 2.2 trc for compatibility.
A better discriminator might be global edits vs local edits, with local edits being things like retouching specific parts of the image to make desired changes, and one could argue that local edits are "more fake" than global edits, but it still depends on a thousand factors, most importantly intent.
"Fake" images are images with intent to deceive. By that definition, even an image that came straight out of the camera can be "fake" if it's showing something other than what it's purported to (e.g. a real photo of police violence but with a label saying it's in a different country is a fake photo).
What most people think when they say "fake", though, is a photo that has had filters applied, which makes zero sense. As the post shows, all photos have filters applied. We should get over that specific editing process, it's no more fake than anything else.
Even that isn't all that clear-cut. Is noise removal a local edit? It only touches some pixels, but obviously, that's a silly take.
Is automated dust removal still global? The same idea, just a bit more selective. But then, what about automated skin blemish removal? Depth map + relighting, de-hazing, or fake bokeh? I think that modern image processing techniques really blur the distinction here because many edits that would previously need to be done by hand are now a keypress away.
Intent is the defining factor, as you note, but intent is... often hazy. If you dial down the exposure to make the photo more dramatic / more sinister, you're manipulating emotions too. And that's something that's perfectly OK in photojournalism. Adding or remove elements for dramatic effect? Not so much.
The only process in the article that involves nearby pixels is to combine R G and B (and other G) into one screen pixel. (In principle these could be mapped to subpixels.) Everything fancier than that can be reasonably called some fake cosmetic bullshit.
Regarding sharpening and optical stuff, many modern camera lenses are built with the expectation that some of their optical properties will be easy to correct for in software, allowing the manufacturer to optimize for other properties.
If you want reality, go there in person and stop looking at photos. Viewing imagery is a fundamentally different type of experience.
We’ve had similar debates about art using miniatures and lens distortions versus photos since photography was invented — and digital editing fell on the lens trick and miniature side of the issue.
Portrait photography -- no, people don't look like that in real life with skin flaws edited out
Landscape photography -- no, the landscapes don't look like that 99% of the time, the photographer picks the 1% of the time when it looks surreal
Staged photography -- no, it didn't really happen
Street photography -- a lot of it is staged spontaneously
Product photography -- no, they don't look like that in normal lighting
A lot of armchair critics on the internet who only go out to their local park at high noon will say they look fake but they're not.
Artists, who use these tools with clear vision and intent to achieve specific goals, strangely never have this problem.
For example, I took a picture of my dog at the dog park the other day. I didn’t notice when framing the picture but on review at home, right smack in the middle of the lower 3rd of the photo and conveniently positioned to have your eyes led there by my dog’s pose and snout direction, was a giant, old, crusty turd. Once you noticed it, it was very hard to not see it anymore. So I broke out the photo editing tools and used some auto retouching tool to remove the turd. And lucky for me since the ground was mulch, the tool did a fantastic job of blending it out, and if I didn’t tell you it had been retouched, you wouldn’t know.
Is that a fake image? The subject of the photo was my dog. The purpose of the photo was to capture my dog doing something entertaining. When I was watching the scene with my own human eyes I didn’t see the turd. Nor was capturing the turd in the photo intended or essential to capturing what I wanted to capture. But I did use some generative tool (algorithmic or AI I couldn’t say) to convincingly replace the turd with more mulch. So does doing that make the image fake? I would argue no. If you ask me what the photo is, I say it’s a photo of my dog. The edit does not change my dog, nor change the surrounding to make the dog appear somewhere else or to make the dog appear to be doing something they weren’t doing were you there to witness it yourself. I do not intend the photo to be used as a demonstration of how clean that particular dog park is or was on that day, or even to be a photo representing that dog park at all. My dog happened to be in that locale when they did something I wanted a picture of. So to me that picture is no more fake than any other picture in my library. But a pure “differentiate on the tools” analysis says it is a fake image, content that wasn’t captured by the sensor is now in the image and content that was captured no longer is. Fake image then right?
I think the OP has it right, the intent of your use of the tool (and its effect) matters more than what specific tool you used.
And if shot composition doesn't make it fake, what if I cropped the photo after the fact? I'm removing something the camera captured to better make the picture what I "felt like it should be" just using the removal tool. That's functionally no different from framing the shot differently, but it's modifying the actual captured image.
If we decide that removal, whether by framing or by post-hoc cropping is still "real" and it's the use of a tool that adds something that wasn't there, would the same apply to just cutting a square out of the photo without cropping the rest of the frame? A transparent square would be an interesting artistic choice for sure, but does that then get into the realm of "fake"? What if the square is black or white? Is adding a clearly "post process censor bar" crossing a line into making the photo "fake"?
If those are fine, it's the "adding content that looks like it should be there" is the problem, does that mean that dust or hair removal to remove something that was on the lens make it fake since that would also have to generate what the computer thinks is behind that hair or dust speck?
For what it's worth, I don't think there is a hard line here, like I said intent matters. But I do think that figuring out where ones personal and general lines are and when they might move that line or not is an interesting thought experiment.
What about this? https://news.ycombinator.com/item?id=35107601
I downsized it to 170x170 pixels
For example see
You can try to guess the location of edges to enhance them after upscaling, but it's guessing, and when the source is a 170x170 moon photo a big proportion of the guessing will inevitably be wrong.
I zoomed in on the monitor showing that image and, guess what, again you see slapped on detail, even in the parts I explicitly clipped (made completely 100% white):
The first words I said were that Samsung probably did this
From one of the comments there:
> When people take a picture on the moon, they want a cool looking picture of the moon, and every time I have take a picture of the moon, on what is a couple of year old phone which had the best camera set up at the time, it looks awful, because the dynamic range and zoom level required is just not at all what smart phones are good at.
> Hence they solved the problem and gave you your picture of the moon. Which is what you wanted, not a scientifically accurate representation of the light being hit by the camera sensor. We had that, it is called 2010.
Where does one draw the line though? This is a kind of lying, regardless of the whole discussion about filters and photos always being an interpretation of raw sensor data and whatnot.
Again, where does one draw the line? The person taking a snapshot of the moon expects a correlation between the data captured by the sensor and whatever they end up showing their friends. What if the camera only acknowledged "ok, this user is trying to photograph the moon" and replaced ALL of the sensor data with a library image of the moon it has stored in its memory? Would this be authentic or fake? It's certainly A photo of the moon, just not a photo taken with the current camera. But the user believes it's taken with their camera.
I think this is lying.
You can look it up because it's published on the web but IIRC it's generally what you'd expect. It's okay to do whole-image processing where all pixels have the same algorithm applied like the basic brightness, contrast, color, tint, gamma, levels, cropping, scaling, etc filters that have been standard for decades. The usual debayering and color space conversions are also fine. Selectively removing, adding or changing only some pixels or objects is generally not okay for journalistic purposes. Obviously, per-object AI enhancement of the type many mobile phones and social media apps apply by default don't meet such standards.
Maybe don't bring that up, unless you want your boss to think you're a tedious blowhard.
Filters themselves don't make it fake, just like words themselves don't make something a lie. How the filters and words are used, whether they bring us closer or further from some truth, is what makes the difference.
Photos implicitly convey, usually, 'this is what you would see if you were there'. Obviously filters can help with that, as in the OP, or hurt.
https://www.theguardian.com/australia-news/2020/sep/13/pictu...
Raw data requires interpretation, no argument there.
But when AI starts making stuff up out of nowhere, it becomes a problem. Again, some degree of making up stuff is ok, but AI often crosses the line. When it diverges enough from what was captured by the sensor, it crosses firmly into "fake" territory.
For example if you add snow to a shot with masking or generative AI. It's fake because the real life experience was not actually snowing. You can't just hallucinate a major part of the image - that counts as fake to me. A major departure from the reality of the scene. Many other types of edits don't have this property because they are mostly based on the reality of what occurred.
I think for me this comes from an intrinsic valuing of the act/craft of photography, in the physical sense. Once an image is too digitally manipulated then it's less photography and more digital art.
So there are levels of image processing, and it would be wrong to dump them all in the same category.
Ditto for black and white photos. Your visual perception has pretty high dynamic range. Not least because your eyes move and your brain creates a representation that gives you the illusion of higher dynamic range than what your eyes can actually deliver. So when you want to represent it using a technology that can only give you a fraction of the dynamic range you (or your camera) can see, you sometimes make local’ish edits (eg create a mask with brush or gradients to lighten or darken regions)
Ansel Adams did a lot of dodging and burning in his prints. Some of the more famous ones are very obvious in terms of having been “processed” during the exposure of the print.
I see this as overcoming the limitations in conveying what your eyes/brain will see when using the limited capabilities of camera/screen/print. It is local’ish edits, but the intent isn’t so much to deceive as it is to nudge information into a range where it can be seen/understood.
i.e. Camera+Lens+ISO+SS+FStop+FL+TC (If present)+Filter (If present). Add focus distance if being super duper proper.
And some of that is to help at least provide the right requirements to try to recreate.
I mean to some degree, human perception is a hallucination of reality. It is well known by magicians that if you know the small region of space that a person is focusing on, then you can totally change other areas of the scene without the person noticing.
A fun tangent on the "green cast" mentioned in the post: the reason the Bayer pattern is RGGB (50% green) isn't just about color balance, but spatial resolution. The human eye is most sensitive to green light, so that channel effectively carries the majority of the luminance (brightness/detail) data. In many advanced demosaicing algorithms, the pipeline actually reconstructs the green channel first to get a high-resolution luminance map, and then interpolates the red/blue signals—which act more like "color difference" layers—on top of it. We can get away with this because the human visual system is much more forgiving of low-resolution color data than it is of low-resolution brightness data. It’s the same psycho-visual principle that justifies 4:2:0 chroma subsampling in video compression.
Also, for anyone interested in how deep the rabbit hole goes, looking at the source code for dcraw (or libraw) is a rite of passage. It’s impressive how many edge cases exist just to interpret the "raw" voltages from different sensor manufacturers.
When I worked at Amazon on the Kindle Special Offers team (ads on your eink Kindle while it was sleeping), the first implementation of auto-generated ads was by someone who didn't know that properly converting RGB to grayscale was a smidge more complicated than just averaging the RGB channels. So for ~6 months in 2015ish, you may have seen a bunch of ads that looked pretty rough. I think I just needed to add a flag to the FFmpeg call to get it to convert RGB to luminance before mapping it to the 4-bit grayscale needed.
I remember trying out some of the home-made methods while I was implementing a creative work section for a school assignment. It’s surprising how "flat" the basic average looks until you actually respect the coefficients (usually some flavor of 0.21R + 0.72G + 0.07B). I bet it's even more apparent in a 4-bit display.
It is necessary that it was precisely defined because of the requirements of backwards-compatible color transmission, basically they treated B&W (technically monochrome) pictures like how B&W film and videotubes treated them: great in green, average in red, and poorly in blue.
A bit unrelated: pre-color transition, the makeups used are actually slightly greenish too (which appears nicely in monochrome).
Page 5 has:
Eq' = 0.41 (Eb' - Ey') + 0.48 (Er' - Ey')
Ei' = -0.27(Eb' - Ey') + 0.74 (Er' - Ey')
Ey' = 0.30Er' + 0.59Eg' + 0.11Eb'
PAL and SECAM use different color primaries than the original NTSC, and a different white, which lead to different coefficients.
However, the original color primaries and white used by NTSC had become obsolete very quickly so they no longer corresponded with what the TV sets could actually reproduce.
Eventually even for NTSC a set of primary colors was used that was close to that of PAL/SECAM, which was standardized by SMPTE in 1987. The NTSC broadcast signal continued to use the original formula, for backwards compatibility, but the equipment processed the colors according to the updated primaries.
In 1990, Rec. 709 has standardized a set of primaries intermediate between those of PAL/SECAM and of SMPTE, which was later also adopted by sRGB.
... okay, technically PAL and SECAM too, but only in audio (analogue Zweikanalton versus digital NICAM), bandwidth placement (channel plan and relative placement of audio and video signals, and, uhm, teletext) and, uhm, teletext standard (French Antiope versus Britain's Teletext and Fastext).
I went to a photoshop conference. There was a session on converting color to black and white. Basically at the end the presenter said you try a bunch of ways and pick the one that looks best.
(people there were really looking for the “one true way”)
I shot a lot of black and white film in college for our paper. One of my obsolete skills was thinking how an image would look in black and white while shooting, though I never understood the people who could look at a scene and decide to use a red filter..
I am shooting a lot of 120-format Ilford HP5+ these days. It's a different pace, a different way of thinking about the craft.
Dark skies and dramatic clouds!
https://i.ibb.co/0RQmbBhJ/05.jpg
(shot on Rollei Superpan with a red filter and developed at home)
As far as ads go that's not bad IMO)
All for ads which are only visible when you aren't using the device anyway. Don't like them? Then buy other devices, pay to have them removed, get a cover to hide them, or just store it with the screen facing down when you aren't using it.
The JPEGs cameras produce are heavily processed, and they are emphatically NOT "original". Taking manual control of that process to produce an alternative JPEG with different curves, mappings, calibrations, is not a crime.
https://puri.sm/posts/librem-5-photo-processing-tutorial/
Which is a pleasant read, and I like the pictures. Has the Librem 5's automatic JPEG output improved since you wrote the post about photography in Croatia (https://dosowisko.net/l5/photos/)?
It's still pretty basic compared to hardware accelerated state-of-the-art, but I think it produces decent output in a fraction of a second on the device itself, which isn't exactly a powerhouse: https://social.librem.one/@dos/115091388610379313
Before that I had an app for offline processing that was calling darktable-cli on the phone, but it took about 30 seconds to process a single photo with it :)
I know my Sony gear can't call out to AI because the WIFI sucks like every other Sony product and barely works inside my house, but also I know the first ILC manufacturer that tries to put AI right into RAW files is probably the first to leave part of the photography market.
That said I'm a purist to the point where I always offer RAWs for my work [0] and don't do any photoshop/etc. D/A, horizon, bright adjust/crop to taste.
Where phones can possibly do better is the smaller size and true MP structure of a cell phone camera sensor, makes it easier to handle things like motion blur. and rolling shutter.
But, I have yet to see anything that gets closer to an ILC for true quality than the decade+ old pureview cameras on Nokia cameras, probably partially because they often had sensors large enough.
There's only so much computation can do to simulate true physics.
[0] - I've found people -like- that. TBH, it helps that I tend to work cheap or for barter type jobs in that scene, however it winds up being something where I've gotten repeat work because they found me and a 'photoshop person' was cheaper than getting an AIO pro.
if you take that away, a picture is not very interesting, it's hyperrealistic so not super creative a lot of the time (compared to eg paintings), & it doesn't even require the mastery of other mediums to get hyperrealistism
I can't see infrared.
But they don't see it as IR. Instead, this infrared information just kind of irrevocably leaks into the RGB channels that we do perceive. With the unmodified camera on my Samsung phone, IR shows up kind of purple-ish. Which is... well... it's fake. Making invisible IR into visible purple is an artificially-produced artifact of the process that results in me being able to see things that are normally ~impossible for me to observe with my eyeballs.
When you generate your own "original" images using your digital camera(s), do you use an external IR filter? Or are you satisfied with knowing that the results are fake?
Your Samsung phone probably has the green filter of its bayer matrix that blocks IR better than the blue and red ones.
Here's a random spectral sensitivity for a silicon sensor:
https://encrypted-tbn0.gstatic.com/images?q=tbn:ANd9GcRkffHX...
>There’s nothing that happens when you adjust the contrast or white balance in editing software that the camera hasn’t done under the hood. The edited image isn’t “faker” then the original: they are different renditions of the same data.
But this is wrong. My not-too-exotic 9-year-old camera has a bunch of settings which affect the resulting image quite a bit. Without going into "picture styles", or "recipes", or whatever they're called these days, I can alter saturation, contrast, and white balance (I can even tell it to add a fixed alteration to the auto WB and tell it to "keep warm colors"). And all these settings will alter how the in-camera produced JPEG will look, no external editing required at all.
So if two people are sitting in the same spot with the same camera, who's to say they both set them up identically? And if they didn't, which produces the "non-processed" one?
I think the point is that the public doesn't really understand how these things work. Even without going to the lengths described by another commenter (local adjust so that there appears to be a ray of light in that particular spot, remove things, etc), just playing with the curves will make people think "it's processed". And what I described above is precisely what the camera itself does. So why is there a difference if I do it manually after the fact or if I tell the camera to do it for me?
Photographs can drop a lot of the perspective, feeling and colour you experience when you’re there. When you take a picture of a slope on a mountain for example (on a ski piste for example), it always looks much less impressive and steep on a phone camera. Same with colours. You can be watching an amazing scene in the mountains, but when you take a photo with most cameras, the colours are more dull, and it just looks flatter. If a filter enhances it and makes it feel as vibrant as the real life view, I’d argue you are making it more realistic.
https://www.winecountry.camera/blog/2021/11/1/moonrise-80-ye...
Most great photos have mediocre and uninteresting subjects. It’s all in the decisions the photographer makes about how to render the final image.
And then there's all the nonsense BigTech enables out of the box today with automated AI touch ups. That definitely qualifies as fakery although the end result may be visually pleasing and some people might find it desirable.
I used to have a high resolution phone camera from a cheaper phone and then later switched to an iPhone. The latter produced much nicer pictures, my old phone just produces very flat-looking pictures.
People say that the iPhone camera automatically edits the images to look better. And in a way I notice that too. But that’s the wrong way of looking at it; the more-edited picture from the iPhone actually corrresponds more to my perception when I’m actually looking at the scene. The white of the snow and glaciers and the deep blue sky really does look amazing in real life, and when my old phone captured it into a flat and disappointing looking photo with less postprocessing than an iPhone, it genuinely failed to capture what I can see with my eyes. And the more vibrant post processed colours of an iPhone really do look more like what I think I’m looking at.
this is totally out of my own self-interest, no problems with its content
also your question implies a bad assumption even if you disclaim it. if you don't want to imply a bad assumption the way to do that is to not say the words, not disclaim them
as for the "assumption" bit, yeah fair enough. was just curious of AI usage online, this wasn't meant to be a dig at anyone as i know people use it for translations, cleaning up prose etc
I'm imagining a sort of Logan's Run-like scifi setup where only people with a documented em dash before November 30, 2022 are left allowed to write.
At least Robespierre needed two sentences before condemning a man. Now the mob is lynching people on the basis of a single glyph.
I have actually been deliberately modifying my long-time writing style and use of punctuation to look less like an LLM. I'm not sure how I feel about this.
But now, likewise, having to bail on emdashes. My last differentiator is that I always close set the emdash—no spaces on either side, whereas ChatGPT typically opens them (AP Style).
Russians use this for at 20 years
what's so special about green? oh so just because our eyes are more sensitive to green we should dedicate double the area to green in camera sensors? i mean, probably yes. but still. (⩺_⩹)
That's presumably what they mean. It's more or less true, except the color in question is at the green / yellow transition.
See e.g. https://s3-us-west-2.amazonaws.com/courses-images-archive-re...
Yes, that's what I meant, and it's definitely true.
For instance, the Leica M series have specific monochrome versions with huge resolutions and better monochrome rendering.
You can also modify some cameras and remove the filter, but the results usually need processing. A side effect is that the now exposed sensor is more sensitive to both ends of the spectrum.
Why is YUV or other luminance-chrominance color spaces important for a RGB input? Because many processing steps and encoders, work in YUV colorspaces. This wasn't really covered in the article.
To truly record an appearance without reference to the sensory system of our species, you would need to encode the full electromagnetic spectrum from each point. Even then, you would still need to decide on a cutoff for the spectrum.
...and hope that nobody ever told you about coherence phenomena.
There is no such thing as “unprocessed” data, at least that we can perceive.
And different films and photo papers can have totally different looks, defined by the chemistry of the manufacturer and however _they_ want things to look.
You’re right about Ansel Adams. He “dodged and burned” extensively (lightened and darkened areas when printing.) Photoshop kept the dodge and burn names on some tools for a while.
https://m.youtube.com/watch?v=IoCtni-WWVs
When we printed for our college paper we had a dial that could adjust the printed contrast a bit of our black and white “multigrade” paper (it added red light). People would mess with the processing to get different results too (cold/ sepia toned). It was hard to get exactly what you wanted and I kind of see why digital took over.
One might argue that there, many of the processing choices are being made by the film manufacturer, in the sensitizing dyes being used, etc.
They were super careful to maintain the look across the transition from film to digital capture. Families display multiple years of school photos next to each other and they wanted a consistent look.
Sometimes the processing has only the goal to compensate the defects of the image sensor and of the optical elements, in order to obtain the most accurate information about the light originally coming from the scene.
Other times the goal of the processing is just to obtain an image that appears best to the photographer, for some reason.
For casual photographers, the latter goal is typical, but in scientific or technical applications the former goal is frequently encountered.
Ideally, a "raw" image format is one where the differences between it and the original image are well characterized and there are no additional unknown image changes done for an "artistic" effect, in order to allow processing when having either one of the previously enumerated goals.
You have layers of substrate with silver halides, made sensitive to different frequency ranges with sensitizing dyes, crystallized into silver halide crystals, rather than a regular grid of pixels; you take a photo that is not an image, but a collection of specks of metallic silver. Through a series of chemical reactions, you develop those specks. Differences in chemistry, in temperatures, in agitation, in the film, all affect what for digital images is described as processing. Then in printing, you have a similar process all over again.
If anything, one might argue that the digital process allows a more consistent and quantitative understanding of the actual processing being done. Analog film seems like it involves less processing only because, for most people, the processing was always a black box of sending off the film for development and printing.
241 more comments available on Hacker News