 
 
Fuck a whole bunch of fancy interpolation, just do linear between points.

Switched from libpnm to opencv since the later tik links it anyway

Fuse the MaskGain.cpp example with render.c from the 20220707 tik... 


Tik.cpp has the command line parsing in it;
Add a -mMASKNAME.png
Add a -kFNFILE.fn  (Why k?, f,g,m are taken)
Let the command line shitshow continue
Can use the parsers from my code


- Do I need WaveRGB or WavePNM?
- bounded by the case wavePNM on 616 and break on 727
	- main change at 647-660 inside the wavepnm case
	
	
						for (where=0; where<xyc; ++where) {
							/* Add portion of previous sample */
							if (fstart < fstop) {
								register double w = fstop - fstart;
								SUM(where) += (gamma[ref[where]] * w);
							}

							/* Add portion of gap as a slope */
							if (gstart < gstop) {
								/* Compute gap end values, then average * wgap to get area */
								register double vstart = (ref[where] * (1.0 - wstart)) + (gamma[p[where]] * wstart);
								register double vstop = (ref[where] * (1.0 - wstop)) + (gamma[p[where]] * wstop);
								SUM(where) += (wgap * ((vstart + vstop) / 2.0));
							}
						}
		
	- also the segment from 712-721 for residual parts
	 
	 				for (where=0; where<xyc; ++where) {
					register double f = now - wf;
					register double w = STOP - ((f < START) ? START : f);

					if (w < 0) w = 0;
					SUM(where) = (SUM(where) + (gamma[ref[where]] * w)) * DIVBY;
					if (e_gamma != 1) {
						SUM(where) = pow(SUM(where), invgamma);
					}
				}
	- this thing gets hacked ... do I literally just need to stub it into DIVBY?
	- Then the command line needs to get integrated...oh no, it's already awful.
	

	
######## 20240326 Findings! ########
	- The path that seems to actually be in use is WaveRGB
		- proven by printf at 227
	- Gain Lookups inserted:
		- 226. for (i=0; i<wavec; ++i) SUM(off+i) += (gamma[ref[off+i]] * w);
		- 242. SUM(off+i) += (wgap * ((vstart + vstop) / 2.0));
		- 260. SUM(off) = (SUM(off) + (gamma[ref[off]] * w)) * DIVBY;
		
	

#	- IF we were using wavepnm - AND I JUST PROVED WE AREN'T - Gain lookups at three places!
#		- 651. SUM(where) += (gamma[ref[where]] * w);  //W becomes a lookup
#		- 660. SUM(where) += (wgap * ((vstart + vstop) / 2.0)); //wgap becomes a lookup
#		- 718. SUM(where) = (SUM(where) + (gamma[ref[where]] * w)) * DIVBY; // Add in some guesses for data after the last sample, w becomes a lookup
	
	- I need to not return ints for gains. 
		- I do want to be outside [-1,1] to support nondestructive amplification 
		- Probably doubles.
		- That's gonna be a lot of casting to make sure things promote at the right time, holy fuck.
		
	
	- Normalization problem
		- You should clip at 0 and maxval because there isn't an "anti-image" 
		- Psycho option: negative red is added to the other channels.  HELL NO.
		- Norm option: Could rescale so most-negative is 0 as a tone mapping thing.  Potentially makes unexpected shit, no. 
########
	
Hank's TIK code does time per-sample-interval.  My code does per-nanosecond value.

There's a terrible option of extracting nanosecond sized slices from the sample data - no. 

There is an option of breaking it into "Next Linear Interval" - each section is broken on edges in either the input samples or the function control points.
	- I'm having trouble convincing myself this will do the right thing when there is a slope over a nontrivial slice since it'll only do midpoint averages over each slice... but it'll be close enough. 
	
	- I need to know where the next control point in the function and the next sample are
	[START STOP]
	
	what I really need is to return a gain that is "the average gain between now and the next control point, where the next control point might be a weight function, might be a edge of a sample, and might be the edge of an output frame. 
	
	interval t has [start,stop,qual,divby,

	
### Integration Notes 20240626###
Hank's code still thinks about exposing FRAMES with a start/stop and a stride defined by fps and shutter angle and ...NO WE JUST FOLLOW THE FUNCTIONS.
There is going to be an interaction problem between the -a, -b, -f, -n, and -t flags and the function-driven exposure with -m and -k.
... Maybe I just need to hack it to derive START and STOP from the function specs?
	- Finding the first time where a function has a non-zero value isn't adequate because interpolation before it.
	- Finding one control point _before_ the earliest non-0 in a spec file would?

- Seems to be a gamma problem?

### Support Tools ###
- Gonna need to write a grapher for functions.
	X Gnuplot?
	- Matplotlib
	
### improvement possibilities:  ###
- Linear approximation causes discontinuities because the edges don't necessarily have the same derivative.  Could do curvilinear interpolation?
- Could use the temporal super-resolution tricks from that paper. 
	- Would want to super-resolution the input stream to get finer increments THEN do the segmentation.  Absolutely not. 
	- Would create a per-site time basis instead of a per-global-sample time basis, and that's a math nightmare. 
	- Would have to then basically sort the superresolution output by location then time. 
	- The slight saving grace is that the individual sites would be independent and therefore work in parallel. 
- Time-Varying masks
	- the same object recognition being used for fake depth by blurring and such could do dynamic masks
* Performance
	- Use a search that ... isn't a linear scan everywhere
	- Applicative caching because gains will tend to be the same for long runs
	- This thing is embarrassingly parallel along several axes
		- process different image areas/tiles on different cores
		- Process different color ranges
		- Process different time ranges...
	- Didn't: Harder to hack, no reward.




---

Start documenting this piece of shit I haven't even succeeded in putting together: 
	
A later more sophisticated prototype of the non-uniform exposure was constructed by attaching an implementation of the gain masking and function specification system to the rendering code of a version of the TIK Sources which were cleaned up and adapted to use some OpenCV library code in mid-2022.  

This prototype has been termed NUTIK (Non Uniform TIK), and adds support for arguments specifying masks to separate regions for different integration functions, function spec files to specify lists of integration functions mapped to the mask values, and a considerable amount of internal machinery in the rendering process to apply the new features.  


These new features are activated with added command line flags; Mask files are specified to nutik with the command line option \texttt{-mMaskFile.pgm} and Function files are specified to nutik with the command line option \texttt{-kFnFile.fn}.


The mask specification has remained more or less stable through all the different iterations of the Non-Uniform exposure code: an 8-bit PGM image (as PGM reading has been deferred to libraries, ASCII encoded P2 or a binary encoded P5 are both fine), where each level is associated with a different function.  
This allows for up to 256 unique functions to be specified for different portions of the frame.  
This mask must be of of the same spatial resolution as the input steam such that each site has a clearly defined gain. 
The mask can be created - essentially- by drawing over an exposed frame in a conventional image editor from the source to isolate desired correctly exposed features, and filling it with a number between 0 and 255 which is also used to mark the function that produced the desired exposure. 


The function specification in this version uses both a different scheme for representing the functions and a different syntax for encoding them.
After some experimentation with the original Octave prototype, one of the lessons is that sophisticated functions are not particularly desirable - the simple case is camera-like exposures specified by boxcar functions whose length in X is the duration of the exposure and height in Y is the gain.  
The decision to use sophisticated interpolation through a series of control points in the first version makes representing boxcar functions difficult, as near-vertical features in higher-order functions or splines tends to produce overshoot, undershoot, and rolled over corners. 
Therefore, this version performs simple linear interpolation between points, optimizing for the simple case of roughly boxcar functions, and allowing more complicated functions to be described with more points, potentially by generating them in other software.
This is also rather easier to write standalone code for.  

 

The encoding used in this version specifies one function per line, in the format:

MN{[t0:g0],[t1:g1],...,[tn:gn]}

where N is a value 0-255 for the corresponding mask value, each tn is a time in ns from the start of the capture, and each gn is a gain to be applied at that point, both expressed as a floating point number.
Lines not starting with an M are discarded as comments.

The points \textit{MUST} be specified in increasing order by time, such that they describe a function - consideration was given to sorting in software, but even early experiments made it clear that it is better to make mistakes obvious rather than potentially surprising behavior.  
A simple example of a problem that is easy to make a specification file that will result in unexpected behavior is a missing digit - as times are in nS, it is easy for a human to miss a trailing 0 and place a control point a power of ten earlier than intended, resulting in an unwanted long exposure.  
It is preferable that this be an error than an unexpected early control point that wildly changes the function.
One convenience feature which was added to the NUTIK implementation is that times before the first or after the last control point are assumed to have the value of the first or last control point. 
This is again an optimization for the common case in which there are only contributions integrated from relatively brief windows, surrounded by extended regions of no contribution before and after. 

An EBNF grammar for the specification format is supplied in listing !!!! REF TO LISTING OF EBNF !!!. 

This format is relatively simple to hand-write, simple to parse, and those same properties also make it relatively straightforward to programmatically generate, with an eye toward using it as an interface for potential future interaction with higher level software.

There are two implementations of this exposure function specification file format, one integrated into the NUTIK codebase for generating exposures, and another in a support tool for plotting functions.  

FnPlotter.py, listed in appendix !!! REF TO FnPlotter.py!!! is a relatively simple Python script which plots all the functions specified in the file as a set of stacked graphs.  
This is accomplished by parsing exposure function spec files into a data structure compatible with the MatPlotLib \cite{Hunter:2007} plotting library, then generating a subplot for each specified function.
This support tool allows for easy visualization of described functions in a human-readable form. 

Another feature considered but rejected for this prototype is the design of an integrated format that contains both an exposure mask, the function specifications, and perhaps even the scene data to be rendered from in a single file.  
The idea of an integrated format seems appealing as an interchange format and operator convenience, but this argument doesn't seem to hold much weight at this time.  
There is not a great deal of value in being able to easily move and handle exposure specifications; generally those are expected to working intermediate data - in the sense that a Photoshop psd or GIMP xcf file is typically not the final product for easy exchange, but a format for intermediate work before a final image is rendered.  
Conversely, sticking to plaintext and pgm images enables easy manipulation of the files with existing tools during experimentation is desirable. 
The utility of directly handling the native formats in existing text editors, image editors, scripts, and other established software, and leveraging existing user knowledge of that software is quite high. 

--

NUTIK uses the existing tik infrastructure to render a \texttt{.tik} file from an input video stream, then performs function-controlled rendering of output frames from that \texttt{.tik} file.  

Contributions from the input .tik stream are segmented on the edges of input data, output interval, and on the control points, and added to the output image weighted by the length of the contribution interval and the average function-specified gain during that interval.

The list of edges on which contributions to an output are segmented is: beginning of output interval, end of output interval, beginning of sample interval, end of sample interval, beginning of gap between samples, end of gap between samples, and integration function control point.  

Sample intervals and gap intervals can be interrupted by control points in the gain function, and are split on any control point, then handled as two or more intervals with different average gain.  

