BasicOptimizer.scala:89 executed in 68.45 seconds (1.024 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 2928190426081
Reset training subject: 2928662851876
Adding measurement 500f70b2 to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-5.011370365980435;dx=-8.176368556187422E-5
New Minimum: -5.011370365980435 > -5.011376804033381
WOLFE (weak): th(2.154434690031884)=-5.011376804033381; dx=-8.18056278508937E-5 evalInputDelta=6.438052945867412E-6
New Minimum: -5.011376804033381 > -5.0113846077228335
WOLFE (weak): th(4.308869380063768)=-5.0113846077228335; dx=-8.177852141413239E-5 evalInputDelta=1.4241742398724E-5
New Minimum: -5.0113846077228335 > -5.011412115631238
WOLFE (weak): th(12.926608140191302)=-5.011412115631238; dx=-8.176305003020358E-5 evalInputDelta=4.174965080316895E-5
New Minimum: -5.011412115631238 > -5.011536387066037
WOLFE (weak): th(51.70643256076521)=-5.011536387066037; dx=-8.179607786564883E-5 evalInputDelta=1.6602108560270779E-4
New Minimum: -5.011536387066037 > -5.012216801457595
WOLFE (weak): th(258.53216280382605)=-5.012216801457595; dx=-8.195150436693471E-5 evalInputDelta=8.464354771602345E-4
New Minimum: -5.012216801457595 > -5.016580532865363
WOLFE (weak): th(1551.1929768229563)=-5.016580532865363; dx=-8.153333400951851E-5 evalInputDelta=0.005210166884928036
New Minimum: -5.016580532865363 > -5.04854918780357
WOLFE (weak): th(10858.350837760694)=-5.04854918780357; dx=-7.63723774573883E-5 evalInputDelta=0.0371788218231357
New Minimum: -5.04854918780357 > -5.341255215817535
END: th(86866.80670208555)=-5.341255215817535; dx=-4.598118522417206E-5 evalInputDelta=0.3298848498371001
Fitness changed from -5.011370365980435 to -5.341255215817535
Iteration 1 complete. Error: -5.341255215817535 Total: 5.9045; Orientation: 0.0165; Line Search: 4.4906
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 485d5297 to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-5.341255215817535;dx=-7.915003551479054E-5
New Minimum: -5.341255215817535 > -5.749502178980432
END: th(187148.86177126726)=-5.749502178980432; dx=-2.4721449484596833E-5 evalInputDelta=0.4082469631628971
Fitness changed from -5.341255215817535 to -5.749502178980432
Iteration 2 complete. Error: -5.749502178980432 Total: 1.5052; Orientation: 0.0157; Line Search: 1.0156
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 1b7d6b90 to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-5.749502178980432;dx=-6.763458601848754E-5
New Minimum: -5.749502178980432 > -7.020262248668792
END: th(403200.00000000006)=-7.020262248668792; dx=-9.58004649521087E-6 evalInputDelta=1.27076006968836
Fitness changed from -5.749502178980432 to -7.020262248668792
Iteration 3 complete. Error: -7.020262248668792 Total: 1.5802; Orientation: 0.0159; Line Search: 1.0173
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 783444c7 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.532e+03, gradient 8.465e-03, dot -0.234; [3231abe1-c673-4907-9bf8-96cf2128b511 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -7.020262248668792, -5.749502178980432, -5.341255215817535, -5.011370365980435
LBFGS Accumulation History: 3 points
Removed measurement 783444c7 to history. Total: 3
th(0)=-7.020262248668792;dx=-7.155903786168035E-5
New Minimum: -7.020262248668792 > -8.055082782323934
END: th(868668.0670208557)=-8.055082782323934; dx=-2.3612936807738973E-6 evalInputDelta=1.0348205336551421
Fitness changed from -7.020262248668792 to -8.055082782323934
Iteration 4 complete. Error: -8.055082782323934 Total: 1.9354; Orientation: 0.4159; Line Search: 1.0169
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 6597e4e5 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.074e+03, gradient 7.369e-03, dot -0.208; [3231abe1-c673-4907-9bf8-96cf2128b511 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -8.055082782323934, -5.749502178980432, -5.341255215817535, -5.011370365980435
LBFGS Accumulation History: 3 points
Removed measurement 6597e4e5 to history. Total: 3
th(0)=-8.055082782323934;dx=-5.4083662172777145E-5
New Minimum: -8.055082782323934 > -8.895349888778377
END: th(1871488.617712673)=-8.895349888778377; dx=-8.718059466020599E-7 evalInputDelta=0.8402671064544425
Fitness changed from -8.055082782323934 to -8.895349888778377
Iteration 5 complete. Error: -8.895349888778377 Total: 1.9038; Orientation: 0.4293; Line Search: 0.9982
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 599cef87 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.081e+03, gradient 5.522e-03, dot -0.151; [3231abe1-c673-4907-9bf8-96cf2128b511 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -8.895349888778377, -5.749502178980432, -5.341255215817535, -5.011370365980435
LBFGS Accumulation History: 3 points
Removed measurement 599cef87 to history. Total: 3
th(0)=-8.895349888778377;dx=-3.032353146448314E-5
New Minimum: -8.895349888778377 > -9.693166305346324
END: th(4032000.0000000014)=-9.693166305346324; dx=-2.5992931204006716E-7 evalInputDelta=0.7978164165679473
Fitness changed from -8.895349888778377 to -9.693166305346324
Iteration 6 complete. Error: -9.693166305346324 Total: 2.6362; Orientation: 1.1233; Line Search: 1.0410
Adding measurement a16f590 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.903e+03, gradient 3.888e-03, dot -0.111; [3231abe1-c673-4907-9bf8-96cf2128b511 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -9.693166305346324, -5.749502178980432, -5.341255215817535, -5.011370365980435
LBFGS Accumulation History: 3 points
Removed measurement a16f590 to history. Total: 3
th(0)=-9.693166305346324;dx=-1.5002863292189108E-5
Armijo: th(8686680.670208558)=-9.662866514911734; dx=-8.940433034623286E-8 evalInputDelta=-0.03029979043459008
New Minimum: -9.693166305346324 > -9.9735577712009
END: th(4343340.335104279)=-9.9735577712009; dx=-9.710842923855873E-7 evalInputDelta=0.2803914658545761
Fitness changed from -9.693166305346324 to -9.9735577712009
Iteration 7 complete. Error: -9.9735577712009 Total: 2.4044; Orientation: 0.4107; Line Search: 1.5468
Adding measurement 4bc922d to history. Total: 3
Rejected: LBFGS Orientation magnitude: 6.578e+03, gradient 3.303e-03, dot -0.076; [3231abe1-c673-4907-9bf8-96cf2128b511 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -9.9735577712009, -5.749502178980432, -5.341255215817535, -5.011370365980435
LBFGS Accumulation History: 3 points
Removed measurement 4bc922d to history. Total: 3
th(0)=-9.9735577712009;dx=-1.0853519045393757E-5
New Minimum: -9.9735577712009 > -10.004758910422744
END: th(9357443.088563366)=-10.004758910422744; dx=-1.5273470022913112E-7 evalInputDelta=0.031201139221844088
Fitness changed from -9.9735577712009 to -10.004758910422744
Iteration 8 complete. Error: -10.004758910422744 Total: 1.9503; Orientation: 0.4099; Line Search: 1.0708
Adding measurement cd0285b to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.320e+03, gradient 2.560e-03, dot -0.052; [3231abe1-c673-4907-9bf8-96cf2128b511 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -10.004758910422744, -5.749502178980432, -5.341255215817535, -5.011370365980435
LBFGS Accumulation History: 3 points
Removed measurement cd0285b to history. Total: 3
th(0)=-10.004758910422744;dx=-6.5020467530183555E-6
Armijo: th(2.0160000000000007E7)=-9.520807554734853; dx=-2.6803746028402492E-8 evalInputDelta=-0.48395135568789094
Armijo: th(1.0080000000000004E7)=-9.858485028972314; dx=-3.371569319440414E-7 evalInputDelta=-0.14627388145042985
New Minimum: -10.004758910422744 > -10.370846108084445
END: th(3360000.0000000014)=-10.370846108084445; dx=-1.9627570267351215E-6 evalInputDelta=0.3660871976617006
Fitness changed from -10.004758910422744 to -10.370846108084445
Iteration 9 complete. Error: -10.370846108084445 Total
...skipping 7008 bytes...
.0830373945096493E7)=-12.2063352398632; dx=-4.765761578148478E-7 evalInputDelta=0.1177972475240896
Fitness changed from -12.088537992339111 to -12.2063352398632
Iteration 17 complete. Error: -12.2063352398632 Total: 2.4123; Orientation: 0.4050; Line Search: 1.5546
Adding measurement 1eb35026 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.315e+05, gradient 1.866e-03, dot -0.094; [3231abe1-c673-4907-9bf8-96cf2128b511 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -12.2063352398632, -5.749502178980432, -5.341255215817535, -5.011370365980435
LBFGS Accumulation History: 3 points
Removed measurement 1eb35026 to history. Total: 3
th(0)=-12.2063352398632;dx=-3.4635822629518003E-6
Armijo: th(2.3333333333333354E7)=-11.643933614620902; dx=-9.011482425771423E-8 evalInputDelta=-0.5624016252422983
New Minimum: -12.2063352398632 > -12.42777462023488
END: th(1.1666666666666677E7)=-12.42777462023488; dx=-4.477623373298677E-7 evalInputDelta=0.2214393803716792
Fitness changed from -12.2063352398632 to -12.42777462023488
Iteration 18 complete. Error: -12.42777462023488 Total: 2.9944; Orientation: 0.4081; Line Search: 2.1286
Adding measurement 6394c0c7 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.070e+06, gradient 1.791e-03, dot -0.095; [3231abe1-c673-4907-9bf8-96cf2128b511 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -12.42777462023488, -5.749502178980432, -5.341255215817535, -5.011370365980435
LBFGS Accumulation History: 3 points
Removed measurement 6394c0c7 to history. Total: 3
th(0)=-12.42777462023488;dx=-3.1896455375743047E-6
Armijo: th(2.5135071383705333E7)=-11.968499360752231; dx=-7.570612163422425E-8 evalInputDelta=-0.4592752594826486
New Minimum: -12.42777462023488 > -12.910966795006484
END: th(1.2567535691852666E7)=-12.910966795006484; dx=-3.99089500610407E-7 evalInputDelta=0.48319217477160414
Fitness changed from -12.42777462023488 to -12.910966795006484
Iteration 19 complete. Error: -12.910966795006484 Total: 2.4512; Orientation: 0.4237; Line Search: 1.5698
Adding measurement bc9fff4 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.845e+06, gradient 1.714e-03, dot -0.093; [3231abe1-c673-4907-9bf8-96cf2128b511 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -12.910966795006484, -5.749502178980432, -5.341255215817535, -5.011370365980435
LBFGS Accumulation History: 3 points
Removed measurement bc9fff4 to history. Total: 3
th(0)=-12.910966795006484;dx=-2.9225615077333357E-6
Armijo: th(2.7075934862741236E7)=-11.621646815288964; dx=-7.28234409575933E-8 evalInputDelta=-1.2893199797175203
Armijo: th(1.3537967431370618E7)=-12.578466388191632; dx=-3.413159029139607E-7 evalInputDelta=-0.3325004068148516
New Minimum: -12.910966795006484 > -13.09547246523721
END: th(4512655.810456873)=-13.09547246523721; dx=-1.1808748505719442E-6 evalInputDelta=0.18450567023072573
Fitness changed from -12.910966795006484 to -13.09547246523721
Iteration 20 complete. Error: -13.09547246523721 Total: 2.9661; Orientation: 0.4046; Line Search: 2.1007
<a id="p-17"></a>Iteration 20
<a id="p-16"></a>
Adding measurement 7b7d5846 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.834e+05, gradient 1.663e-03, dot -0.118; [3231abe1-c673-4907-9bf8-96cf2128b511 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -13.09547246523721, -5.749502178980432, -5.341255215817535, -5.011370365980435
LBFGS Accumulation History: 3 points
Removed measurement 7b7d5846 to history. Total: 3
th(0)=-13.09547246523721;dx=-2.745520976056744E-6
Armijo: th(9722222.222222233)=-12.737365629196638; dx=-5.492779442346105E-7 evalInputDelta=-0.35810683604057125
Armijo: th(4861111.111111117)=-13.029085554117511; dx=-1.0729736649521793E-6 evalInputDelta=-0.06638691111969841
New Minimum: -13.09547246523721 > -13.139671251431004
END: th(1620370.3703703722)=-13.139671251431004; dx=-1.6946803268782601E-6 evalInputDelta=0.04419878619379425
Fitness changed from -13.09547246523721 to -13.139671251431004
Iteration 21 complete. Error: -13.139671251431004 Total: 3.0532; Orientation: 0.4155; Line Search: 2.1314
Adding measurement 2504aa5c to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.403e+04, gradient 1.653e-03, dot -0.103; [3231abe1-c673-4907-9bf8-96cf2128b511 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -13.139671251431004, -5.749502178980432, -5.341255215817535, -5.011370365980435
LBFGS Accumulation History: 3 points
Removed measurement 2504aa5c to history. Total: 3
th(0)=-13.139671251431004;dx=-2.7208479662561707E-6
New Minimum: -13.139671251431004 > -13.141981827223677
END: th(3490982.1366257416)=-13.141981827223677; dx=-1.2799843811001218E-6 evalInputDelta=0.002310575792673575
Fitness changed from -13.139671251431004 to -13.141981827223677
Iteration 22 complete. Error: -13.141981827223677 Total: 1.9644; Orientation: 0.4536; Line Search: 1.0355
Adding measurement fb434e8 to history. Total: 3
Accepted: LBFGS Orientation magnitude: 7.020e+04, gradient 1.658e-03, dot 0.145; [3231abe1-c673-4907-9bf8-96cf2128b511 = 1.000/1.000e+00]
Constructing line search parameters: LBFGS+Trust
th(0)=-13.141981827223677;dx=-16.856713101456606
Armijo: th(7521093.017428123)=-9.449115846748823; dx=6.378275908326258E-8 evalInputDelta=-3.6928659804748545
Armijo: th(3760546.5087140617)=-9.449188274097207; dx=1.2796180498092318E-7 evalInputDelta=-3.692793553126471
Armijo: th(1253515.5029046873)=-9.449349681619315; dx=3.807209512740166E-7 evalInputDelta=-3.692632145604362
Armijo: th(313378.8757261718)=-9.449369546969951; dx=1.52409630722798E-6 evalInputDelta=-3.6926122802537265
Armijo: th(62675.775145234365)=-9.448796332048223; dx=7.616483625005563E-6 evalInputDelta=-3.693185495175454
Armijo: th(10445.962524205728)=-9.44978091588602; dx=4.582448467733382E-5 evalInputDelta=-3.6922009113376575
Armijo: th(1492.2803606008183)=-9.438291028385803; dx=3.1558826583986125E-4 evalInputDelta=-3.703690798837874
Armijo: th(186.5350450751023)=-9.458558174634685; dx=0.0027216912024937657 evalInputDelta=-3.683423652588992
Armijo: th(20.72611611945581)=-9.706933602198756; dx=0.026012090169700464 evalInputDelta=-3.4350482250249215
Armijo: th(2.072611611945581)=-11.196967611574035; dx=0.028898556973308627 evalInputDelta=-1.9450142156496426
Armijo: th(0.1884192374495983)=-12.782470441316685; dx=-4.798648262028447 evalInputDelta=-0.35951138590699294
Armijo: th(0.015701603120799858)=-13.10402129806451; dx=-8.955690702298098 evalInputDelta=-0.03796052915916803
Armijo: th(0.001207815624676912)=-13.139180157083365; dx=-9.476645921032187 evalInputDelta=-0.0028016701403128508
Armijo: th(8.627254461977944E-5)=-13.141782450429021; dx=-9.511143216607747 evalInputDelta=-1.9937679465620306E-4
Armijo: th(5.751502974651963E-6)=-13.141968733915036; dx=-9.530177638885426 evalInputDelta=-1.3093308641742851E-5
Armijo: th(3.594689359157477E-7)=-13.141981232073567; dx=-9.523717131232171 evalInputDelta=-5.951501105272428E-7
Armijo: th(2.1145231524455745E-8)=-13.141981232073567; dx=-9.510819415288237 evalInputDelta=-5.951501105272428E-7
Armijo: th(1.1747350846919858E-9)=-13.141981827223677; dx=-9.533457242303104 evalInputDelta=0.0
MIN ALPHA (6.182816235220978E-11): th(0.0)=-13.141981827223677
Fitness changed from -13.141981827223677 to -13.141981827223677
Static Iteration Total: 11.4615; Orientation: 0.4086; Line Search: 10.5782
Iteration 23 failed. Error: -13.141981827223677
Previous Error: 0.0 -> -13.141981827223677
Retrying iteration 23
Reset training subject: 2994655463141
Adding measurement 371e942f to history. Total: 0
LBFGS Accumulation History: 1 points
th(0)=-13.141981827223677;dx=-2.727343849882408E-6
END: th(2.1544346900318838E-10)=-13.141981827223677; dx=-2.0862481830231604E-6 evalInputDelta=0.0
Fitness changed from -13.141981827223677 to -13.141981827223677
Static Iteration Total: 1.9858; Orientation: 0.0167; Line Search: 1.0392
Iteration 24 failed. Error: -13.141981827223677
Previous Error: 0.0 -> -13.141981827223677
Optimization terminated 24
Final threshold in iteration 24: -13.141981827223677 (> -Infinity) after 68.451s (< 3600.000s)
Returns
-13.141981827223677