BasicOptimizer.scala:89 executed in 62.16 seconds (0.751 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 151982811784900
Reset training subject: 151983737768400
Adding measurement d659940 to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-2.1564138403848134;dx=-1.0914959484276126E-4
New Minimum: -2.1564138403848134 > -2.1565997459514707
WOLFE (weak): th(2.154434690031884)=-2.1565997459514707; dx=-1.0754811075352404E-4 evalInputDelta=1.859055666573184E-4
New Minimum: -2.1565997459514707 > -2.1567862119983268
WOLFE (weak): th(4.308869380063768)=-2.1567862119983268; dx=-1.080881034199219E-4 evalInputDelta=3.723716135133337E-4
New Minimum: -2.1567862119983268 > -2.157531503384755
WOLFE (weak): th(12.926608140191302)=-2.157531503384755; dx=-1.083801784162971E-4 evalInputDelta=0.0011176629999414267
New Minimum: -2.157531503384755 > -2.1608862025848428
WOLFE (weak): th(51.70643256076521)=-2.1608862025848428; dx=-1.0811400556143718E-4 evalInputDelta=0.004472362200029334
New Minimum: -2.1608862025848428 > -2.1787128699764073
WOLFE (weak): th(258.53216280382605)=-2.1787128699764073; dx=-1.0720560267250617E-4 evalInputDelta=0.02229902959159391
New Minimum: -2.1787128699764073 > -2.2897375270184863
WOLFE (weak): th(1551.1929768229563)=-2.2897375270184863; dx=-1.0719295251362025E-4 evalInputDelta=0.1333236866336729
New Minimum: -2.2897375270184863 > -3.0412472099460004
END: th(10858.350837760694)=-3.0412472099460004; dx=-9.381272430211311E-5 evalInputDelta=0.884833369561187
Fitness changed from -2.1564138403848134 to -3.0412472099460004
Iteration 1 complete. Error: -3.0412472099460004 Total: 9.6184; Orientation: 0.0158; Line Search: 7.0555
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 2b51053d to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-3.0412472099460004;dx=-9.827908087077911E-5
New Minimum: -3.0412472099460004 > -4.493473830041286
END: th(23393.607721408407)=-4.493473830041286; dx=-6.297697787446532E-5 evalInputDelta=1.4522266200952858
Fitness changed from -3.0412472099460004 to -4.493473830041286
Iteration 2 complete. Error: -4.493473830041286 Total: 2.6218; Orientation: 0.0146; Line Search: 1.6972
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 59df9027 to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-4.493473830041286;dx=-6.868520591314995E-5
New Minimum: -4.493473830041286 > -6.4554602587370695
END: th(50400.00000000001)=-6.4554602587370695; dx=-4.021602561759841E-5 evalInputDelta=1.9619864286957833
Fitness changed from -4.493473830041286 to -6.4554602587370695
Iteration 3 complete. Error: -6.4554602587370695 Total: 2.7102; Orientation: 0.0150; Line Search: 1.7448
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 17efb41d to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.054e+03, gradient 7.897e-03, dot -0.534; [9cce9019-06a9-4ac4-a80e-43e691858664 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -6.4554602587370695, -4.493473830041286, -3.0412472099460004, -2.1564138403848134
LBFGS Accumulation History: 3 points
Removed measurement 17efb41d to history. Total: 3
th(0)=-6.4554602587370695;dx=-6.198242689559649E-5
New Minimum: -6.4554602587370695 > -8.736254583934931
END: th(108583.50837760697)=-8.736254583934931; dx=-1.846325693699609E-5 evalInputDelta=2.2807943251978617
Fitness changed from -6.4554602587370695 to -8.736254583934931
Iteration 4 complete. Error: -8.736254583934931 Total: 3.1563; Orientation: 0.3135; Line Search: 1.9154
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 7f828f7d to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.029e+03, gradient 9.398e-03, dot -0.297; [9cce9019-06a9-4ac4-a80e-43e691858664 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -8.736254583934931, -4.493473830041286, -3.0412472099460004, -2.1564138403848134
LBFGS Accumulation History: 3 points
Removed measurement 7f828f7d to history. Total: 3
th(0)=-8.736254583934931;dx=-8.81565187516466E-5
New Minimum: -8.736254583934931 > -10.562096401801417
END: th(233936.07721408413)=-10.562096401801417; dx=-7.602043403188024E-6 evalInputDelta=1.8258418178664861
Fitness changed from -8.736254583934931 to -10.562096401801417
Iteration 5 complete. Error: -10.562096401801417 Total: 2.9118; Orientation: 0.2718; Line Search: 1.6698
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 288dc441 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.956e+03, gradient 8.441e-03, dot -0.226; [9cce9019-06a9-4ac4-a80e-43e691858664 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -10.562096401801417, -4.493473830041286, -3.0412472099460004, -2.1564138403848134
LBFGS Accumulation History: 3 points
Removed measurement 288dc441 to history. Total: 3
th(0)=-10.562096401801417;dx=-7.115085533774952E-5
New Minimum: -10.562096401801417 > -15.522672880511303
END: th(504000.0000000002)=-15.522672880511303; dx=-2.2622197590588873E-6 evalInputDelta=4.960576478709886
Fitness changed from -10.562096401801417 to -15.522672880511303
Iteration 6 complete. Error: -15.522672880511303 Total: 2.9222; Orientation: 0.2725; Line Search: 1.7262
Adding measurement 1c905c81 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.684e+03, gradient 7.987e-03, dot -0.135; [9cce9019-06a9-4ac4-a80e-43e691858664 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -15.522672880511303, -4.493473830041286, -3.0412472099460004, -2.1564138403848134
LBFGS Accumulation History: 3 points
Removed measurement 1c905c81 to history. Total: 3
th(0)=-15.522672880511303;dx=-6.353960065101387E-5
New Minimum: -15.522672880511303 > -21.33988618848974
WOLF (strong): th(1085835.0837760698)=-21.33988618848974; dx=8.533324837766497E-7 evalInputDelta=5.817213307978436
New Minimum: -21.33988618848974 > -23.657501224040796
END: th(542917.5418880349)=-23.657501224040796; dx=-5.335876507302032E-6 evalInputDelta=8.134828343529493
Fitness changed from -15.522672880511303 to -23.657501224040796
Iteration 7 complete. Error: -23.657501224040796 Total: 3.7672; Orientation: 0.2787; Line Search: 2.7000
Adding measurement ecd5bc1 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 8.391e+03, gradient 7.331e-03, dot -0.186; [9cce9019-06a9-4ac4-a80e-43e691858664 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -23.657501224040796, -4.493473830041286, -3.0412472099460004, -2.1564138403848134
LBFGS Accumulation History: 3 points
Removed measurement ecd5bc1 to history. Total: 3
th(0)=-23.657501224040796;dx=-5.358746302553241E-5
New Minimum: -23.657501224040796 > -29.419300150788736
WOLF (strong): th(1169680.3860704207)=-29.419300150788736; dx=3.1880329793355823E-6 evalInputDelta=5.7617989267479395
New Minimum: -29.419300150788736 > -33.82899989242736
END: th(584840.1930352104)=-33.82899989242736; dx=-6.187891352205732E-6 evalInputDelta=10.171498668386565
Fitness changed from -23.657501224040796 to -33.82899989242736
Iteration 8 complete. Error: -33.82899989242736 Total: 3.6450; Orientation: 0.2639; Line Search: 2.5687
Adding measurement 2456aa9a to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.861e+04, gradient 7.529e-03, dot -0.232; [9cce9019-06a9-4ac4-a80e-43e691858664 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -33.82899989242736, -4.493473830041286, -3.0412472099460004, -2.1564138403848134
LBFGS Accumulation History: 3 points
Removed measurement 2456aa9a to history. Total: 3
th(0)=-33.82899989242736;dx=-5.654328777297814E-5
New Minimum: -33.82899989242736 > -37.262408555757865
WOLF (strong): th(1260000.0000000005)=-37.262408555757865; dx=2.2907876324122545E-6 evalInputDelta=3.4334086633305034
New Minimum: -37.262408555757865 > -42.93556624995163
END: th(630000.0000000002)=-42.93556624995163; dx=-4.375520651959063E-6 evalInputDelta=9.106566357524265
Fitness changed from -33.82899989242736 to -42.93556624995163
Iteration 9 complete. Error: -42.93556624995163 Total: 3.7273; Orientation: 0.3004; Line Search: 2.6076
Adding measurement 3dfc5ff7 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 6.435e+05, gradient 7.768e-03, dot -0.258; [9cce9019-06a9-4ac4-a80e-43e691858664 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -42.93556624995163, -4.493473830041286, -3.0412472099460004, -2.1564138403848134
LBFGS Accumulation History: 3 points
Removed measurement 3dfc5ff7 to history. Total: 3
th(0)=-42.93556624995163;dx=-5.993362372694467E-5
New Minimum: -42.93556624995163 > -43.46766643992753
WOLF (strong): th(1357293.8547200873)=-43.46766643992753; dx=4.668534920854783E-6 evalInputDelta=0.5321001899758997
New Minimum: -43.46766643992753 > -52.64286607720799
END: th(678646.9273600436)=-52.64286607720799; dx=-1.0156415507931083E-6 evalInputDelta=9.707299827256364
Fitness changed from -42.93556624995163 to -52.64286607720799
Iteration 10 complete. Error: -52.64286607720799 Total: 3.7298; Orientation: 0.2650; Line Search: 2.6695
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Adding measurement 7fc7514e to history. Total: 3
Accepted: LBFGS Orientation magnitude: 6.156e+04, gradient 8.153e-03, dot 0.264; [9cce9019-06a9-4ac4-a80e-43e691858664 = 1.000/1.000e+00]
Constructing line search parameters: LBFGS+Trust
th(0)=-52.64286607720799;dx=-132.7917981756699
Armijo: th(1462100.482588026)=-24.398932138862154; dx=1.6240548924552074E-5 evalInputDelta=-28.243933938345837
Armijo: th(731050.241294013)=-24.3994884531272; dx=3.264601846253864E-5 evalInputDelta=-28.24337762408079
Armijo: th(243683.413764671)=-24.404539429904016; dx=9.780605504126051E-5 evalInputDelta=-28.238326647303975
Armijo: th(60920.85344116775)=-24.411676659470825; dx=3.9129580292287847E-4 evalInputDelta=-28.231189417737166
Armijo: th(12184.17068823355)=-24.42026893358432; dx=0.0019555505333825034 evalInputDelta=-28.22259714362367
Armijo: th(2030.6951147055915)=-24.45773063366193; dx=0.011713336092425464 evalInputDelta=-28.18513544354606
Armijo: th(290.0993021007988)=-24.54101572105123; dx=0.08156907862565034 evalInputDelta=-28.10185035615676
Armijo: th(36.26241276259985)=-25.00082731909399; dx=0.6471961674987184 evalInputDelta=-27.642038758114
Armijo: th(4.0291569736222055)=-28.72731171702587; dx=5.130890741803756 evalInputDelta=-23.91555436018212
Armijo: th(0.40291569736222055)=-51.20791756652267; dx=13.31555134394155 evalInputDelta=-1.4349485106853237
New Minimum: -52.64286607720799 > -55.49308993259683
END: th(0.03662869976020187)=-55.49308993259683; dx=-64.03839213771451 evalInputDelta=2.8502238553888404
Fitness changed from -52.64286607720799 to -55.49308993259683
Iteration 11 complete. Error: -55.49308993259683 Total: 11.5822; Orientation: 0.2646; Line Search: 10.3654
Adding measurement 5eee3793 to history. Total: 4
Rejected: LBFGS Orientation magnitude: 4.470e+04, gradient 8.219e-03, dot -0.169; [9cce9019-06a9-4ac4-a80e-43e691858664 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -55.49308993259683, -52.64286607720799, -4.493473830041286, -3.0412472099460004, -2.1564138403848134
Rejected: LBFGS Orientation magnitude: 7.660e+04, gradient 8.219e-03, dot -0.185; [9cce9019-06a9-4ac4-a80e-43e691858664 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -55.49308993259683, -52.64286607720799, -4.493473830041286, -3.0412472099460004
LBFGS Accumulation History: 3 points
Removed measurement 5eee3793 to history. Total: 4
Removed measurement 7fc7514e to history. Total: 3
th(0)=-55.49308993259683;dx=-6.719192304003541E-5
New Minimum: -55.49308993259683 > -55.49309648804948
END: th(0.07891414141414145)=-55.49309648804948; dx=-5.518742960771749E-5 evalInputDelta=6.555452650047755E-6
Fitness changed from -55.49308993259683 to -55.49309648804948
Iteration 12 complete. Error: -55.49309648804948 Total: 3.1442; Orientation: 0.6282; Line Search: 1.7130
Adding measurement 22b57291 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.532e+05, gradient 8.209e-03, dot -0.211; [9cce9019-06a9-4ac4-a80e-43e691858664 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -55.49309648804948, -4.493473830041286, -3.0412472099460004, -2.1564138403848134
LBFGS Accumulation History: 3 points
Removed measurement 22b57291 to history. Total: 3
th(0)=-55.49309648804948;dx=-6.730015328797455E-5
New Minimum: -55.49309648804948 > -55.493106321227
END: th(0.1700153637967081)=-55.493106321227; dx=-5.5165016544710586E-5 evalInputDelta=9.833177522011738E-6
Fitness changed from -55.49309648804948 to -55.493106321227
Iteration 13 complete. Error: -55.493106321227 Total: 2.7513; Orientation: 0.2629; Line Search: 1.6720
Adding measurement 46ecbd73 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.568e+05, gradient 8.210e-03, dot -0.211; [9cce9019-06a9-4ac4-a80e-43e691858664 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -55.493106321227, -4.493473830041286, -3.0412472099460004, -2.1564138403848134
LBFGS Accumulation History: 3 points
Removed measurement 46ecbd73 to history. Total: 3
th(0)=-55.493106321227;dx=-6.722341721731968E-5
New Minimum: -55.493106321227 > -55.49312598757682
END: th(0.36628699760201877)=-55.49312598757682; dx=-5.5189670043816786E-5 evalInputDelta=1.966634981442894E-5
Fitness changed from -55.493106321227 to -55.49312598757682
Iteration 14 complete. Error: -55.49312598757682 Total: 2.9322; Orientation: 0.2705; Line Search: 1.8272
Adding measurement 750b8efb to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.465e+05, gradient 8.210e-03, dot -0.210; [9cce9019-06a9-4ac4-a80e-43e691858664 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -55.49312598757682, -4.493473830041286, -3.0412472099460004, -2.1564138403848134
LBFGS Accumulation History: 3 points
Removed measurement 750b8efb to history. Total: 3
th(0)=-55.49312598757682;dx=-6.716604627814158E-5
New Minimum: -55.49312598757682 > -55.49316532025554
END: th(0.7891414141414147)=-55.49316532025554; dx=-5.5109778839943056E-5 evalInputDelta=3.9332678724690595E-5
Fitness changed from -55.49312598757682 to -55.49316532025554
Iteration 15 complete. Error: -55.49316532025554 Total: 2.8317; Orientation: 0.2704; Line Search: 1.7657
<a id="p-15"></a>Iteration 15
<a id="p-14"></a>
Final threshold in iteration 16: -55.49316532025554 (> -Infinity) after 62.157s (< 5400.000s)
Returns
-55.49316532025554