BasicOptimizer.scala:89 executed in 148.67 seconds (2.417 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 461600298982300
Reset training subject: 461603234213300
Corrupt weights measurement
LBFGS Accumulation History: 0 points
Constructing line search parameters: GD+Trust
New Minimum: -101.29617593437341 > -101.31974950275678
th(0)=-101.31974950275678;dx=-1.5233857861295843E-5
New Minimum: -101.31974950275678 > -101.3197769234441
END: th(2.154434690031884)=-101.3197769234441; dx=-1.2335039506699327E-5 evalInputDelta=2.7420687317203374E-5
Fitness changed from -101.29617593437341 to -101.3197769234441
Iteration 1 complete. Error: -101.3197769234441 Total: 14.1009; Orientation: 0.0676; Line Search: 5.7664
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement b71d055 to history. Total: 0
LBFGS Accumulation History: 1 points
th(0)=-101.3197769234441;dx=-1.2365790206536698E-5
New Minimum: -101.3197769234441 > -101.31984273306338
WOLFE (weak): th(4.641588833612779)=-101.31984273306338; dx=-1.2347207579146562E-5 evalInputDelta=6.580961928648321E-5
New Minimum: -101.31984273306338 > -101.31990305851019
WOLFE (weak): th(9.283177667225559)=-101.31990305851019; dx=-1.2334701780876694E-5 evalInputDelta=1.261350660968219E-4
New Minimum: -101.31990305851019 > -101.32014435993821
WOLFE (weak): th(27.849533001676676)=-101.32014435993821; dx=-1.2331688651626596E-5 evalInputDelta=3.6743649411619117E-4
New Minimum: -101.32014435993821 > -101.32121924113606
WOLFE (weak): th(111.3981320067067)=-101.32121924113606; dx=-1.2326911098142373E-5 evalInputDelta=0.0014423176919677871
New Minimum: -101.32121924113606 > -101.32698830660026
WOLFE (weak): th(556.9906600335335)=-101.32698830660026; dx=-1.2311780034897422E-5 evalInputDelta=0.007211383156160878
New Minimum: -101.32698830660026 > -101.36300463588749
WOLFE (weak): th(3341.9439602012008)=-101.36300463588749; dx=-1.231978542770048E-5 evalInputDelta=0.043227712443396626
New Minimum: -101.36300463588749 > -101.62182619407237
WOLFE (weak): th(23393.607721408407)=-101.62182619407237; dx=-1.2259442205266429E-5 evalInputDelta=0.3020492706282738
New Minimum: -101.62182619407237 > -103.69464331251686
WOLFE (weak): th(187148.86177126726)=-103.69464331251686; dx=-1.1840415081562272E-5 evalInputDelta=2.374866389072764
New Minimum: -103.69464331251686 > -118.97369879489315
END: th(1684339.7559414054)=-118.97369879489315; dx=-7.69671086863522E-6 evalInputDelta=17.653921871449057
Fitness changed from -101.3197769234441 to -118.97369879489315
Iteration 2 complete. Error: -118.97369879489315 Total: 31.3603; Orientation: 0.0900; Line Search: 27.9274
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 607b60ff to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-118.97369879489315;dx=-7.863245209059613E-6
New Minimum: -118.97369879489315 > -138.66065768763164
END: th(3628800.0000000005)=-138.66065768763164; dx=-3.3144126656068373E-6 evalInputDelta=19.686958892738488
Fitness changed from -118.97369879489315 to -138.66065768763164
Iteration 3 complete. Error: -138.66065768763164 Total: 9.3484; Orientation: 0.1025; Line Search: 5.8473
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 785fa6e4 to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-138.66065768763164;dx=-5.081591269043061E-6
New Minimum: -138.66065768763164 > -156.153897463326
END: th(7818012.603187701)=-156.153897463326; dx=-6.474229436741831E-7 evalInputDelta=17.49323977569435
Fitness changed from -138.66065768763164 to -156.153897463326
Iteration 4 complete. Error: -156.153897463326 Total: 9.0911; Orientation: 0.0865; Line Search: 5.6129
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 5c80378a to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.702e+04, gradient 2.505e-03, dot -0.481; [d9248dcd-2c98-4941-acf1-4108093fb5ee = 1.000/1.000e+00]
Orientation rejected. Popping history element from -156.153897463326, -138.66065768763164, -118.97369879489315, -101.3197769234441
LBFGS Accumulation History: 3 points
Removed measurement 5c80378a to history. Total: 3
th(0)=-156.153897463326;dx=-6.277139013537653E-6
Armijo: th(1.6843397559414055E7)=-149.2730648400267; dx=1.753226760701034E-6 evalInputDelta=-6.880832623299284
New Minimum: -156.153897463326 > -166.62723910315287
WOLF (strong): th(8421698.779707028)=-166.62723910315287; dx=1.3751907188085041E-6 evalInputDelta=10.473341639826884
New Minimum: -166.62723910315287 > -168.3967381109267
END: th(2807232.926569009)=-168.3967381109267; dx=-2.211051306231206E-6 evalInputDelta=12.242840647600701
Fitness changed from -156.153897463326 to -168.3967381109267
Iteration 5 complete. Error: -168.3967381109267 Total: 16.4166; Orientation: 1.9177; Line Search: 11.0503
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 665c4f02 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.819e+04, gradient 1.754e-03, dot -0.650; [d9248dcd-2c98-4941-acf1-4108093fb5ee = 1.000/1.000e+00]
Orientation rejected. Popping history element from -168.3967381109267, -138.66065768763164, -118.97369879489315, -101.3197769234441
LBFGS Accumulation History: 3 points
Removed measurement 665c4f02 to history. Total: 3
th(0)=-168.3967381109267;dx=-3.0708991869401307E-6
New Minimum: -168.3967381109267 > -179.23026956370947
END: th(6048000.000000001)=-179.23026956370947; dx=-8.984794989983256E-7 evalInputDelta=10.833531452782779
Fitness changed from -168.3967381109267 to -179.23026956370947
Iteration 6 complete. Error: -179.23026956370947 Total: 11.0830; Orientation: 1.7882; Line Search: 5.5913
Adding measurement 59ef1ae5 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.553e+04, gradient 1.919e-03, dot -0.560; [d9248dcd-2c98-4941-acf1-4108093fb5ee = 1.000/1.000e+00]
Orientation rejected. Popping history element from -179.23026956370947, -138.66065768763164, -118.97369879489315, -101.3197769234441
LBFGS Accumulation History: 3 points
Removed measurement 59ef1ae5 to history. Total: 3
th(0)=-179.23026956370947;dx=-3.6778972341019284E-6
New Minimum: -179.23026956370947 > -181.25714739492923
WOLF (strong): th(1.3030021005312836E7)=-181.25714739492923; dx=9.25035048996662E-7 evalInputDelta=2.026877831219764
New Minimum: -181.25714739492923 > -187.0162552772257
WOLF (strong): th(6515010.502656418)=-187.0162552772257; dx=2.3875449009246934E-7 evalInputDelta=7.785985713516226
END: th(2171670.167552139)=-185.1911990607761; dx=-1.7341600240873065E-6 evalInputDelta=5.960929497066644
Fitness changed from -179.23026956370947 to -187.0162552772257
Iteration 7 complete. Error: -187.0162552772257 Total: 16.3559; Orientation: 1.7814; Line Search: 11.8268
Adding measurement 452c67d9 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.868e+04, gradient 2.127e-03, dot -0.469; [d9248dcd-2c98-4941-acf1-4108093fb5ee = 1.000/1.000e+00]
Orientation rejected. Popping history element from -187.0162552772257, -138.66065768763164, -118.97369879489315, -101.3197769234441
LBFGS Accumulation History: 3 points
Removed measurement 452c67d9 to history. Total: 3
th(0)=-187.0162552772257;dx=-4.504115833481685E-6
New Minimum: -187.0162552772257 > -197.43878523613543
END: th(4678721.544281682)=-197.43878523613543; dx=-1.3341285273996807E-8 evalInputDelta=10.422529958909735
Fitness changed from -187.0162552772257 to -197.43878523613543
Iteration 8 complete. Error: -197.43878523613543 Total: 10.2700; Orientation: 1.9124; Line Search: 5.7165
Adding measurement 121bb344 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.754e+04, gradient 1.710e-03, dot -0.554; [d9248dcd-2c98-4941-acf1-4108093fb5ee = 1.000/1.000e+00]
Orientation rejected. Popping history element from -197.43878523613543, -138.66065768763164, -118.97369879489315, -101.3197769234441
LBFGS Accumulation History: 3 points
Removed measurement 121bb344 to history. Total: 3
th(0)=-197.43878523613543;dx=-2.9042743366754486E-6
New Minimum: -197.43878523613543 > -201.53470200914904
WOLF (strong): th(1.0080000000000002E7)=-201.53470200914904; dx=8.014023384185481E-7 evalInputDelta=4.095916773013613
New Minimum: -201.53470200914904 > -204.47079055619
END: th(5040000.000000001)=-204.47079055619; dx=-1.8495525427933016E-7 evalInputDelta=7.032005320054566
Fitness changed from -197.43878523613543 to -204.47079055619
Iteration 9 complete. Error: -204.47079055619 Total: 13.3636; Orientation: 1.7128; Line Search: 8.6120
Adding measurement 5d7add7a to history. Total: 3
Rejected: LBFGS Orientation magnitude: 6.383e+04, gradient 1.684e-03, dot -0.545; [d9248dcd-2c98-4941-acf1-4108093fb5ee = 1.000/1.000e+00]
Orientation rejected. Popping history element from -204.47079055619, -138.66065768763164, -118.97369879489315, -101.3197769234441
LBFGS Accumulation History: 3 points
Removed measurement 5d7add7a to history. Total: 3
th(0)=-204.47079055619;dx=-2.8371440887063655E-6
New Minimum: -204.47079055619 > -206.41961099497348
WOLF (strong): th(1.0858350837760696E7)=-206.41961099497348; dx=9.33986324651088E-7 evalInputDelta=1.9488204387834855
New Minimum: -206.41961099497348 > -210.7794448022781
WOLF (strong): th(5429175.418880348)=-210.7794448022781; dx=9.67232875448177E-8 evalInputDelta=6.3086542460881105
END: th(1809725.1396267826)=-208.6195157778398; dx=-1.6544200155490305E-6 evalInputDelta=4.148725221649812
Fitness changed from -204.47079055619 to -210.7794448022781
Iteration 10 complete. Error: -210.7794448022781 Total: 16.5947; Orientation: 1.7834; Line Search: 11.7883
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Final threshold in iteration 11: -210.7794448022781 (> -Infinity) after 148.671s (< 5400.000s)
Returns
-210.7794448022781