BasicOptimizer.scala:89 executed in 140.81 seconds (1.544 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 7792579938562
Reset training subject: 7793860949752
Adding measurement 50207ad0 to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-16.753855329089262;dx=-6.346103505930694E-4
New Minimum: -16.753855329089262 > -16.754000601399756
WOLFE (weak): th(2.154434690031884)=-16.754000601399756; dx=-6.365068709110843E-4 evalInputDelta=1.4527231049399347E-4
New Minimum: -16.754000601399756 > -16.754151801854512
WOLFE (weak): th(4.308869380063768)=-16.754151801854512; dx=-6.329535794871951E-4 evalInputDelta=2.96472765249689E-4
New Minimum: -16.754151801854512 > -16.754743990497747
WOLFE (weak): th(12.926608140191302)=-16.754743990497747; dx=-6.356569384548502E-4 evalInputDelta=8.886614084850919E-4
New Minimum: -16.754743990497747 > -16.75741487918185
WOLFE (weak): th(51.70643256076521)=-16.75741487918185; dx=-6.311397137423902E-4 evalInputDelta=0.0035595500925893475
New Minimum: -16.75741487918185 > -16.771369352580084
WOLFE (weak): th(258.53216280382605)=-16.771369352580084; dx=-6.283478125789152E-4 evalInputDelta=0.017514023490821984
New Minimum: -16.771369352580084 > -16.865045602501098
WOLFE (weak): th(1551.1929768229563)=-16.865045602501098; dx=-6.015402117005825E-4 evalInputDelta=0.11119027341183596
New Minimum: -16.865045602501098 > -17.56570387966697
END: th(10858.350837760694)=-17.56570387966697; dx=-4.58715714007652E-4 evalInputDelta=0.8118485505777073
Fitness changed from -16.753855329089262 to -17.56570387966697
Iteration 1 complete. Error: -17.56570387966697 Total: 37.5777; Orientation: 0.0169; Line Search: 33.9713
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement ec14c2 to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-17.56570387966697;dx=-6.458740598168865E-4
New Minimum: -17.56570387966697 > -18.330291769543162
END: th(23393.607721408407)=-18.330291769543162; dx=-3.089812407146917E-4 evalInputDelta=0.7645878898761929
Fitness changed from -17.56570387966697 to -18.330291769543162
Iteration 2 complete. Error: -18.330291769543162 Total: 3.5313; Orientation: 0.0182; Line Search: 2.1987
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 75d13b05 to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-18.330291769543162;dx=-6.243907373855693E-4
New Minimum: -18.330291769543162 > -19.919870547885743
END: th(50400.00000000001)=-19.919870547885743; dx=-1.403028952177075E-4 evalInputDelta=1.589578778342581
Fitness changed from -18.330291769543162 to -19.919870547885743
Iteration 3 complete. Error: -19.919870547885743 Total: 3.6848; Orientation: 0.0233; Line Search: 2.2047
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 391dd3ae to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.676e+03, gradient 2.356e-02, dot -0.241; [e2f9ba34-6cfa-45f5-a319-e839b146621d = 1.000/1.000e+00]
Orientation rejected. Popping history element from -19.919870547885743, -18.330291769543162, -17.56570387966697, -16.753855329089262
LBFGS Accumulation History: 3 points
Removed measurement 391dd3ae to history. Total: 3
th(0)=-19.919870547885743;dx=-5.448615843854053E-4
New Minimum: -19.919870547885743 > -24.385262772266852
END: th(108583.50837760697)=-24.385262772266852; dx=-6.05559683347014E-5 evalInputDelta=4.465392224381109
Fitness changed from -19.919870547885743 to -24.385262772266852
Iteration 4 complete. Error: -24.385262772266852 Total: 4.1870; Orientation: 0.4893; Line Search: 2.3769
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 57bed096 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.436e+03, gradient 2.170e-02, dot -0.160; [e2f9ba34-6cfa-45f5-a319-e839b146621d = 1.000/1.000e+00]
Orientation rejected. Popping history element from -24.385262772266852, -18.330291769543162, -17.56570387966697, -16.753855329089262
LBFGS Accumulation History: 3 points
Removed measurement 57bed096 to history. Total: 3
th(0)=-24.385262772266852;dx=-4.634109487743764E-4
Armijo: th(233936.07721408413)=-24.264904035255203; dx=-2.2704526747679483E-5 evalInputDelta=-0.12035873701164945
New Minimum: -24.385262772266852 > -27.015129975267346
END: th(116968.03860704206)=-27.015129975267346; dx=-8.296181316970114E-5 evalInputDelta=2.6298672030004937
Fitness changed from -24.385262772266852 to -27.015129975267346
Iteration 5 complete. Error: -27.015129975267346 Total: 5.1446; Orientation: 0.4810; Line Search: 3.3168
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 67dfc73e to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.400e+03, gradient 1.978e-02, dot -0.165; [e2f9ba34-6cfa-45f5-a319-e839b146621d = 1.000/1.000e+00]
Orientation rejected. Popping history element from -27.015129975267346, -18.330291769543162, -17.56570387966697, -16.753855329089262
LBFGS Accumulation History: 3 points
Removed measurement 67dfc73e to history. Total: 3
th(0)=-27.015129975267346;dx=-3.82792455720198E-4
New Minimum: -27.015129975267346 > -27.3771300944026
END: th(252000.0000000001)=-27.3771300944026; dx=-3.0580186732386005E-5 evalInputDelta=0.3620001191352529
Fitness changed from -27.015129975267346 to -27.3771300944026
Iteration 6 complete. Error: -27.3771300944026 Total: 4.1753; Orientation: 0.4894; Line Search: 2.2396
Adding measurement 6c2a6b9f to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.887e+03, gradient 1.481e-02, dot -0.130; [e2f9ba34-6cfa-45f5-a319-e839b146621d = 1.000/1.000e+00]
Orientation rejected. Popping history element from -27.3771300944026, -18.330291769543162, -17.56570387966697, -16.753855329089262
LBFGS Accumulation History: 3 points
Removed measurement 6c2a6b9f to history. Total: 3
th(0)=-27.3771300944026;dx=-2.1380727444417425E-4
New Minimum: -27.3771300944026 > -30.288321400874953
END: th(542917.5418880349)=-30.288321400874953; dx=-1.4164190100427662E-5 evalInputDelta=2.911191306472354
Fitness changed from -27.3771300944026 to -30.288321400874953
Iteration 7 complete. Error: -30.288321400874953 Total: 3.8782; Orientation: 0.4851; Line Search: 2.3309
Adding measurement 3d015a6e to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.564e+03, gradient 1.123e-02, dot -0.110; [e2f9ba34-6cfa-45f5-a319-e839b146621d = 1.000/1.000e+00]
Orientation rejected. Popping history element from -30.288321400874953, -18.330291769543162, -17.56570387966697, -16.753855329089262
LBFGS Accumulation History: 3 points
Removed measurement 3d015a6e to history. Total: 3
th(0)=-30.288321400874953;dx=-1.2443280877028453E-4
New Minimum: -30.288321400874953 > -31.18491476651818
END: th(1169680.3860704207)=-31.18491476651818; dx=-5.638156438830026E-6 evalInputDelta=0.896593365643227
Fitness changed from -30.288321400874953 to -31.18491476651818
Iteration 8 complete. Error: -31.18491476651818 Total: 3.7695; Orientation: 0.4816; Line Search: 2.2292
Adding measurement 994db1f to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.887e+03, gradient 7.668e-03, dot -0.099; [e2f9ba34-6cfa-45f5-a319-e839b146621d = 1.000/1.000e+00]
Orientation rejected. Popping history element from -31.18491476651818, -18.330291769543162, -17.56570387966697, -16.753855329089262
LBFGS Accumulation History: 3 points
Removed measurement 994db1f to history. Total: 3
th(0)=-31.18491476651818;dx=-5.7405283426516926E-5
Armijo: th(2520000.000000001)=-31.116772385591723; dx=-2.7885502433688167E-6 evalInputDelta=-0.06814238092645653
New Minimum: -31.18491476651818 > -35.08202776331123
END: th(1260000.0000000005)=-35.08202776331123; dx=-1.0846532556942367E-5 evalInputDelta=3.8971129967930516
Fitness changed from -31.18491476651818 to -35.08202776331123
Iteration 9 complete. Error: -35.08202776331123 Total: 4.9441; Orientation: 0.4960; Line Search: 3.3356
Adding measurement 112e7a4e to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.302e+04, gradient 7.242e-03, dot -0.162; [e2f9ba34-6cfa-45f5-a319-e839b14662
...skipping 2950 bytes...
837E-7 evalInputDelta=-2.3584932629234885
New Minimum: -38.406447024404116 > -42.441210323209106
END: th(6786469.273600437)=-42.441210323209106; dx=-1.5009742710902711E-6 evalInputDelta=4.0347632988049895
Fitness changed from -38.406447024404116 to -42.441210323209106
Iteration 13 complete. Error: -42.441210323209106 Total: 4.9769; Orientation: 0.4843; Line Search: 3.3938
Adding measurement 7ae58f1c to history. Total: 3
Rejected: LBFGS Orientation magnitude: 7.005e+04, gradient 3.646e-03, dot -0.147; [e2f9ba34-6cfa-45f5-a319-e839b146621d = 1.000/1.000e+00]
Orientation rejected. Popping history element from -42.441210323209106, -18.330291769543162, -17.56570387966697, -16.753855329089262
LBFGS Accumulation History: 3 points
Removed measurement 7ae58f1c to history. Total: 3
th(0)=-42.441210323209106;dx=-1.3001172948587529E-5
Armijo: th(1.4621004825880261E7)=-38.057165423491504; dx=-1.731254930144795E-7 evalInputDelta=-4.384044899717601
New Minimum: -42.441210323209106 > -44.89870727139204
END: th(7310502.412940131)=-44.89870727139204; dx=-1.3459351818265393E-6 evalInputDelta=2.4574969481829356
Fitness changed from -42.441210323209106 to -44.89870727139204
Iteration 14 complete. Error: -44.89870727139204 Total: 4.9455; Orientation: 0.4746; Line Search: 3.4163
Adding measurement 2d5790da to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.081e+05, gradient 3.513e-03, dot -0.155; [e2f9ba34-6cfa-45f5-a319-e839b146621d = 1.000/1.000e+00]
Orientation rejected. Popping history element from -44.89870727139204, -18.330291769543162, -17.56570387966697, -16.753855329089262
LBFGS Accumulation History: 3 points
Removed measurement 2d5790da to history. Total: 3
th(0)=-44.89870727139204;dx=-1.2069798026694398E-5
Armijo: th(1.575000000000001E7)=-43.80718124374744; dx=-1.7487889918837792E-7 evalInputDelta=-1.0915260276445977
New Minimum: -44.89870727139204 > -49.907084056628
END: th(7875000.000000005)=-49.907084056628; dx=-1.4035131056728861E-6 evalInputDelta=5.00837678523596
Fitness changed from -44.89870727139204 to -49.907084056628
Iteration 15 complete. Error: -49.907084056628 Total: 4.8793; Orientation: 0.4723; Line Search: 3.3551
<a id="p-15"></a>Iteration 15
<a id="p-14"></a>
Adding measurement 11a59e2c to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.244e+05, gradient 3.454e-03, dot -0.169; [e2f9ba34-6cfa-45f5-a319-e839b146621d = 1.000/1.000e+00]
Orientation rejected. Popping history element from -49.907084056628, -18.330291769543162, -17.56570387966697, -16.753855329089262
LBFGS Accumulation History: 3 points
Removed measurement 11a59e2c to history. Total: 3
th(0)=-49.907084056628;dx=-1.1747678057443273E-5
Armijo: th(1.6966173184001096E7)=-40.51514113194857; dx=-1.3912955973891887E-7 evalInputDelta=-9.391942924679434
Armijo: th(8483086.592000548)=-49.82989287347625; dx=-1.3316915753475013E-6 evalInputDelta=-0.07719118315175422
New Minimum: -49.907084056628 > -52.3363941850219
END: th(2827695.530666849)=-52.3363941850219; dx=-4.628355457133226E-6 evalInputDelta=2.429310128393901
Fitness changed from -49.907084056628 to -52.3363941850219
Iteration 16 complete. Error: -52.3363941850219 Total: 6.4745; Orientation: 0.4861; Line Search: 4.4582
Adding measurement 63f13d6c to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.853e+05, gradient 3.423e-03, dot -0.214; [e2f9ba34-6cfa-45f5-a319-e839b146621d = 1.000/1.000e+00]
Orientation rejected. Popping history element from -52.3363941850219, -18.330291769543162, -17.56570387966697, -16.753855329089262
LBFGS Accumulation History: 3 points
Removed measurement 63f13d6c to history. Total: 3
th(0)=-52.3363941850219;dx=-1.1431798327981939E-5
New Minimum: -52.3363941850219 > -53.41652829871541
END: th(6092085.344116776)=-53.41652829871541; dx=-2.040053770925728E-6 evalInputDelta=1.0801341136935108
Fitness changed from -52.3363941850219 to -53.41652829871541
Iteration 17 complete. Error: -53.41652829871541 Total: 3.8893; Orientation: 0.5891; Line Search: 2.2678
Adding measurement 3dda8981 to history. Total: 3
Accepted: LBFGS Orientation magnitude: 1.616e+05, gradient 3.499e-03, dot 0.200; [e2f9ba34-6cfa-45f5-a319-e839b146621d = 1.000/1.000e+00]
Constructing line search parameters: LBFGS+Trust
th(0)=-53.41652829871541;dx=-113.45927520427666
Armijo: th(1.312500000000001E7)=-44.33359066049217; dx=2.6775103842828083E-7 evalInputDelta=-9.082937638223243
Armijo: th(6562500.000000005)=-44.33359066049217; dx=5.289311923745922E-7 evalInputDelta=-9.082937638223243
Armijo: th(2187500.0000000014)=-44.33359066049217; dx=1.6003776180292496E-6 evalInputDelta=-9.082937638223243
Armijo: th(546875.0000000003)=-44.33359066049217; dx=6.3334756683213345E-6 evalInputDelta=-9.082937638223243
Armijo: th(109375.00000000007)=-44.333350895678976; dx=3.179356629643622E-5 evalInputDelta=-9.083177403036437
Armijo: th(18229.16666666668)=-44.327995063257276; dx=1.900301442146799E-4 evalInputDelta=-9.088533235458137
Armijo: th(2604.1666666666683)=-44.33391557335673; dx=0.001335272797669288 evalInputDelta=-9.082612725358686
Armijo: th(325.52083333333354)=-44.38368875039094; dx=0.010696045920575608 evalInputDelta=-9.032839548324475
Armijo: th(36.1689814814815)=-44.53423155015409; dx=0.09024973278487208 evalInputDelta=-8.882296748561323
Armijo: th(3.61689814814815)=-45.0547815414513; dx=0.6787679662772629 evalInputDelta=-8.361746757264115
Armijo: th(0.32880892255892274)=-50.313604820194286; dx=-7.659858780977977 evalInputDelta=-3.1029234785211273
Armijo: th(0.027400743546576897)=-53.15506024366957; dx=-36.34252924264375 evalInputDelta=-0.2614680550458459
Armijo: th(0.002107749503582838)=-53.3989636631414; dx=-45.35206381769325 evalInputDelta=-0.017564635574011334
Armijo: th(1.5055353597020273E-4)=-53.41531943869236; dx=-46.36904469252124 evalInputDelta=-0.0012088600230555357
Armijo: th(1.0036902398013515E-5)=-53.416446469054044; dx=-46.40922638364839 evalInputDelta=-8.182966136871528E-5
Armijo: th(6.273063998758447E-7)=-53.4165227194243; dx=-46.46053860695175 evalInputDelta=-5.579291112667306E-6
New Minimum: -53.41652829871541 > -53.416530158478984
END: th(3.690037646328498E-8)=-53.416530158478984; dx=-45.68957911663966 evalInputDelta=1.8597635715877914E-6
Fitness changed from -53.41652829871541 to -53.416530158478984
Iteration 18 complete. Error: -53.416530158478984 Total: 22.4606; Orientation: 0.5180; Line Search: 20.8982
Adding measurement 3fc512f0 to history. Total: 4
Rejected: LBFGS Orientation magnitude: 1.928e+06, gradient 3.495e-03, dot -0.162; [e2f9ba34-6cfa-45f5-a319-e839b146621d = 1.000/1.000e+00]
Orientation rejected. Popping history element from -53.416530158478984, -53.41652829871541, -18.330291769543162, -17.56570387966697, -16.753855329089262
Rejected: LBFGS Orientation magnitude: 4.751e+06, gradient 3.495e-03, dot -0.166; [e2f9ba34-6cfa-45f5-a319-e839b146621d = 1.000/1.000e+00]
Orientation rejected. Popping history element from -53.416530158478984, -53.41652829871541, -18.330291769543162, -17.56570387966697
LBFGS Accumulation History: 3 points
Removed measurement 3fc512f0 to history. Total: 4
Removed measurement 3dda8981 to history. Total: 3
th(0)=-53.416530158478984;dx=-1.1997978934619235E-5
END: th(7.94994511277372E-8)=-53.416530158478984; dx=-8.715786540172925E-6 evalInputDelta=0.0
Fitness changed from -53.416530158478984 to -53.416530158478984
Static Iteration Total: 4.5342; Orientation: 1.1329; Line Search: 2.3225
Iteration 19 failed. Error: -53.416530158478984
Previous Error: 0.0 -> -53.416530158478984
Retrying iteration 19
Reset training subject: 7928767123686
Adding measurement 49cb02b7 to history. Total: 0
LBFGS Accumulation History: 1 points
th(0)=-53.416530158478984;dx=-1.1814287335200801E-5
END: th(1.7127637534809138E-7)=-53.416530158478984; dx=-8.694413545432668E-6 evalInputDelta=0.0
Fitness changed from -53.416530158478984 to -53.416530158478984
Static Iteration Total: 4.6265; Orientation: 0.0194; Line Search: 2.3541
Iteration 20 failed. Error: -53.416530158478984
Previous Error: 0.0 -> -53.416530158478984
Optimization terminated 20
Final threshold in iteration 20: -53.416530158478984 (> -Infinity) after 140.814s (< 3600.000s)
Returns
-53.416530158478984