BasicOptimizer.scala:89 executed in 71.49 seconds (0.826 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 139879276502400
Reset training subject: 139880252106300
Adding measurement 6ff11b91 to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-1.3594032098694149;dx=-1.571039267664654E-6
New Minimum: -1.3594032098694149 > -1.3594064033800326
WOLFE (weak): th(2.154434690031884)=-1.3594064033800326; dx=-1.5632047870763992E-6 evalInputDelta=3.1935106177272132E-6
New Minimum: -1.3594064033800326 > -1.3594097718694034
WOLFE (weak): th(4.308869380063768)=-1.3594097718694034; dx=-1.5648765468227111E-6 evalInputDelta=6.561999988496936E-6
New Minimum: -1.3594097718694034 > -1.3594228957743548
WOLFE (weak): th(12.926608140191302)=-1.3594228957743548; dx=-1.5613049341393033E-6 evalInputDelta=1.9685904939947818E-5
New Minimum: -1.3594228957743548 > -1.3594821267557526
WOLFE (weak): th(51.70643256076521)=-1.3594821267557526; dx=-1.5661127537432194E-6 evalInputDelta=7.891688633776539E-5
New Minimum: -1.3594821267557526 > -1.3597978797033266
WOLFE (weak): th(258.53216280382605)=-1.3597978797033266; dx=-1.5559908716753184E-6 evalInputDelta=3.946698339116761E-4
New Minimum: -1.3597978797033266 > -1.3617715656419282
WOLFE (weak): th(1551.1929768229563)=-1.3617715656419282; dx=-1.5595651857638303E-6 evalInputDelta=0.0023683557725133753
New Minimum: -1.3617715656419282 > -1.3759222416308683
WOLFE (weak): th(10858.350837760694)=-1.3759222416308683; dx=-1.5448642127606773E-6 evalInputDelta=0.016519031761453418
New Minimum: -1.3759222416308683 > -1.4902776068757881
WOLFE (weak): th(86866.80670208555)=-1.4902776068757881; dx=-1.5204589754413347E-6 evalInputDelta=0.13087439700637327
New Minimum: -1.4902776068757881 > -2.179706050072849
END: th(781801.26031877)=-2.179706050072849; dx=-7.860191729137123E-7 evalInputDelta=0.820302840203434
Fitness changed from -1.3594032098694149 to -2.179706050072849
Iteration 1 complete. Error: -2.179706050072849 Total: 11.6275; Orientation: 0.0146; Line Search: 8.8314
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 4ba8ffd6 to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-2.179706050072849;dx=-2.0657017859789905E-6
New Minimum: -2.179706050072849 > -3.1925383403938774
END: th(1684339.7559414052)=-3.1925383403938774; dx=-3.500817586552227E-7 evalInputDelta=1.0128322903210285
Fitness changed from -2.179706050072849 to -3.1925383403938774
Iteration 2 complete. Error: -3.1925383403938774 Total: 2.6692; Orientation: 0.0144; Line Search: 1.7006
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 3f1a4d73 to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-3.1925383403938774;dx=-2.0659362716143686E-6
New Minimum: -3.1925383403938774 > -4.723640986422295
END: th(3628800.0)=-4.723640986422295; dx=-1.7286163413352175E-7 evalInputDelta=1.5311026460284172
Fitness changed from -3.1925383403938774 to -4.723640986422295
Iteration 3 complete. Error: -4.723640986422295 Total: 2.6597; Orientation: 0.0147; Line Search: 1.7170
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 6b13bd50 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.146e+04, gradient 1.429e-03, dot -0.257; [14ab1b7b-ad31-48a9-b732-4070ec8505ba = 1.000/1.000e+00]
Orientation rejected. Popping history element from -4.723640986422295, -3.1925383403938774, -2.179706050072849, -1.3594032098694149
LBFGS Accumulation History: 3 points
Removed measurement 6b13bd50 to history. Total: 3
th(0)=-4.723640986422295;dx=-2.018766858946065E-6
New Minimum: -4.723640986422295 > -6.822765351249207
WOLF (strong): th(7818012.6031877)=-6.822765351249207; dx=3.4844803691512793E-8 evalInputDelta=2.0991243648269124
New Minimum: -6.822765351249207 > -7.513804561147265
END: th(3909006.30159385)=-7.513804561147265; dx=-1.5841305712178775E-7 evalInputDelta=2.7901635747249705
Fitness changed from -4.723640986422295 to -7.513804561147265
Iteration 4 complete. Error: -7.513804561147265 Total: 3.9533; Orientation: 0.2828; Line Search: 2.7452
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 5f45b627 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 9.780e+03, gradient 1.400e-03, dot -0.205; [14ab1b7b-ad31-48a9-b732-4070ec8505ba = 1.000/1.000e+00]
Orientation rejected. Popping history element from -7.513804561147265, -3.1925383403938774, -2.179706050072849, -1.3594032098694149
LBFGS Accumulation History: 3 points
Removed measurement 5f45b627 to history. Total: 3
th(0)=-7.513804561147265;dx=-1.941403602445794E-6
New Minimum: -7.513804561147265 > -8.218492259757829
WOLF (strong): th(8421698.779707026)=-8.218492259757829; dx=1.4163398637473948E-7 evalInputDelta=0.7046876986105639
New Minimum: -8.218492259757829 > -10.086651152527997
END: th(4210849.389853513)=-10.086651152527997; dx=-6.650074338750716E-8 evalInputDelta=2.572846591380732
Fitness changed from -7.513804561147265 to -10.086651152527997
Iteration 5 complete. Error: -10.086651152527997 Total: 3.8105; Orientation: 0.2854; Line Search: 2.5915
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 4fdcee63 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.785e+04, gradient 1.520e-03, dot -0.228; [14ab1b7b-ad31-48a9-b732-4070ec8505ba = 1.000/1.000e+00]
Orientation rejected. Popping history element from -10.086651152527997, -3.1925383403938774, -2.179706050072849, -1.3594032098694149
LBFGS Accumulation History: 3 points
Removed measurement 4fdcee63 to history. Total: 3
th(0)=-10.086651152527997;dx=-2.299758588607716E-6
Armijo: th(9072000.0)=-9.651041008935579; dx=2.2092420380275694E-7 evalInputDelta=-0.43561014359241845
New Minimum: -10.086651152527997 > -12.396862957403963
WOLF (strong): th(4536000.0)=-12.396862957403963; dx=9.166152153837926E-8 evalInputDelta=2.3102118048759657
New Minimum: -12.396862957403963 > -12.587026057931805
END: th(1512000.0)=-12.587026057931805; dx=-8.668861566695883E-7 evalInputDelta=2.500374905403808
Fitness changed from -10.086651152527997 to -12.587026057931805
Iteration 6 complete. Error: -12.587026057931805 Total: 4.6312; Orientation: 0.2838; Line Search: 3.4243
Adding measurement 604081dd to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.593e+04, gradient 1.254e-03, dot -0.287; [14ab1b7b-ad31-48a9-b732-4070ec8505ba = 1.000/1.000e+00]
Orientation rejected. Popping history element from -12.587026057931805, -3.1925383403938774, -2.179706050072849, -1.3594032098694149
LBFGS Accumulation History: 3 points
Removed measurement 604081dd to history. Total: 3
th(0)=-12.587026057931805;dx=-1.5605457252338712E-6
New Minimum: -12.587026057931805 > -14.848974475000087
END: th(3257505.2513282085)=-14.848974475000087; dx=-4.173769756822429E-7 evalInputDelta=2.2619484170682824
Fitness changed from -12.587026057931805 to -14.848974475000087
Iteration 7 complete. Error: -14.848974475000087 Total: 2.9518; Orientation: 0.3929; Line Search: 1.7643
Adding measurement 5820d0d2 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.185e+04, gradient 1.436e-03, dot -0.314; [14ab1b7b-ad31-48a9-b732-4070ec8505ba = 1.000/1.000e+00]
Orientation rejected. Popping history element from -14.848974475000087, -3.1925383403938774, -2.179706050072849, -1.3594032098694149
LBFGS Accumulation History: 3 points
Removed measurement 5820d0d2 to history. Total: 3
th(0)=-14.848974475000087;dx=-2.052526381581002E-6
Armijo: th(7018082.316422523)=-14.346010591605147; dx=1.7887518959209886E-7 evalInputDelta=-0.5029638833949406
New Minimum: -14.848974475000087 > -16.33645500456979
END: th(3509041.1582112615)=-16.33645500456979; dx=-2.6864753175514064E-8 evalInputDelta=1.4874805295697016
Fitness changed from -14.848974475000087 to -16.33645500456979
Iteration 8 complete. Error: -16.33645500456979 Total: 3.6023; Orientation: 0.2779; Line Search: 2.5316
Adding measurement 5407ed48 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.977e+04, gradient 1.582e-03, dot -0.253; [14ab1b7b-ad31-48a9-b732-4070ec8505ba = 1.000/1.000e+00]
Orientation rejected. Popping history element from -16.33645500456979, -3.1925383403938774, -2.179706050072849, -1.3594032098694149
LBFGS Accumulation History: 3 points
Removed measurement 5407ed48 to history. Total: 3
th(0)=-16.33645500456979;dx=-2.492617911766343E-6
Armijo: th(7560000.000000002)=-14.447005838829606; dx=4.164810454813722E-7 evalInputDelta=-1.8894491657401833
New Minimum: -16.33645500456979 > -17.921062556014363
WOLF (strong): th(3780000.000000001)=-17.921062556014363; dx=2.7765531155870097E-7 evalInputDelta=1.5846075514445737
New Minimum: -17.921062556014363 > -18.431147952438607
END: th(1260000.0000000002)=-18.431147952438607; dx=-9.443691603676446E-7 evalInputDelta=2.0946929478688183
Fitness changed from -16.33645500456979 to -18.431147952438607
Iteration 9 complete. Error: -18.431147952438607 Total: 4.5414; Orientation: 0.2830; Line Search: 3.4566
Adding measurement 6be26434 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.275e+05, gradient 1.219e-03, dot -0.381; [14ab1b7b-ad31-48a9-b732-4070ec8505ba = 1.000/1.000e+00]
Orientation rejected. Popping history element from -18.431147952438607, -3.1925383403938774, -2.179706050072849, -1.3594032098694149
LBFGS Accumulation History: 3 points
Removed measurement 6be26434 to history. Total: 3
th(0)=-18.431147952438607;dx=-1.4714926362554073E-6
New Minimum: -18.431147952438607 > -20.260942730192514
END: th(2714587.709440174)=-20.260942730192514; dx=-5.061390354048137E-7 evalInputDelta=1.8297947777539072
Fitness changed from -18.431147952438607 to -20.260942730192514
Iteration 10 complete. Error: -20.260942730192514 Total: 2.9978; Orientation: 0.2649; Line Search: 1.7175
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Adding measurement 7fbe8d85 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.705e+05, gradient 1.340e-03, dot -0.383; [14ab1b7b-ad31-48a9-b732-4070ec8505ba = 1.000/1.000e+00]
Orientation rejected. Popping history element from -20.260942730192514, -3.1925383403938774, -2.179706050072849, -1.3594032098694149
LBFGS Accumulation History: 3 points
Removed measurement 7fbe8d85 to history. Total: 3
th(0)=-20.260942730192514;dx=-1.7890057258733889E-6
New Minimum: -20.260942730192514 > -20.411739512100418
WOLF (strong): th(5848401.930352103)=-20.411739512100418; dx=2.281446447540673E-8 evalInputDelta=0.15079678190790347
New Minimum: -20.411739512100418 > -21.421323984356423
END: th(2924200.9651760515)=-21.421323984356423; dx=-1.745819687728548E-7 evalInputDelta=1.1603812541639087
Fitness changed from -20.260942730192514 to -21.421323984356423
Iteration 11 complete. Error: -21.421323984356423 Total: 3.8492; Orientation: 0.2786; Line Search: 2.5952
Adding measurement 154ed788 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.425e+06, gradient 1.463e-03, dot -0.346; [14ab1b7b-ad31-48a9-b732-4070ec8505ba = 1.000/1.000e+00]
Orientation rejected. Popping history element from -21.421323984356423, -3.1925383403938774, -2.179706050072849, -1.3594032098694149
LBFGS Accumulation History: 3 points
Removed measurement 154ed788 to history. Total: 3
th(0)=-21.421323984356423;dx=-2.1281733759687604E-6
Armijo: th(6300000.000000002)=-20.260139461323263; dx=3.0334227983406214E-7 evalInputDelta=-1.1611845230331603
New Minimum: -21.421323984356423 > -22.635301346340118
WOLF (strong): th(3150000.000000001)=-22.635301346340118; dx=1.4952697914815198E-7 evalInputDelta=1.2139773619836944
New Minimum: -22.635301346340118 > -22.844411038598782
END: th(1050000.0000000002)=-22.844411038598782; dx=-8.656785725396573E-7 evalInputDelta=1.4230870542423588
Fitness changed from -21.421323984356423 to -22.844411038598782
Iteration 12 complete. Error: -22.844411038598782 Total: 5.7314; Orientation: 0.2982; Line Search: 4.5678
Adding measurement 3a4bf50f to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.660e+06, gradient 1.157e-03, dot -0.441; [14ab1b7b-ad31-48a9-b732-4070ec8505ba = 1.000/1.000e+00]
Orientation rejected. Popping history element from -22.844411038598782, -3.1925383403938774, -2.179706050072849, -1.3594032098694149
LBFGS Accumulation History: 3 points
Removed measurement 3a4bf50f to history. Total: 3
th(0)=-22.844411038598782;dx=-1.3250923576104312E-6
New Minimum: -22.844411038598782 > -24.18273346739466
END: th(2262156.4245334785)=-24.18273346739466; dx=-5.506145877231395E-7 evalInputDelta=1.3383224287958768
Fitness changed from -22.844411038598782 to -24.18273346739466
Iteration 13 complete. Error: -24.18273346739466 Total: 3.4746; Orientation: 0.5076; Line Search: 1.7866
Adding measurement 27c03adc to history. Total: 3
Accepted: LBFGS Orientation magnitude: 4.371e+04, gradient 1.236e-03, dot 0.566; [14ab1b7b-ad31-48a9-b732-4070ec8505ba = 1.000/1.000e+00]
Constructing line search parameters: LBFGS+Trust
th(0)=-24.18273346739466;dx=-30.64423055569549
Armijo: th(4873668.275293419)=-10.505605118211445; dx=9.907469149180508E-7 evalInputDelta=-13.677128349183214
Armijo: th(2436834.1376467096)=-10.505605118211445; dx=1.9771832049093546E-6 evalInputDelta=-13.677128349183214
Armijo: th(812278.0458822366)=-10.505605118211445; dx=5.951064789013079E-6 evalInputDelta=-13.677128349183214
Armijo: th(203069.51147055914)=-10.505591713569807; dx=2.3796426701129617E-5 evalInputDelta=-13.677141753824852
Armijo: th(40613.90229411183)=-10.50446928587166; dx=1.1925090926360172E-4 evalInputDelta=-13.678264181522998
Armijo: th(6768.9837156853055)=-10.506280400186103; dx=7.153367979820443E-4 evalInputDelta=-13.676453067208556
Armijo: th(966.9976736693294)=-10.526428685061022; dx=0.004979562314912045 evalInputDelta=-13.656304782333637
Armijo: th(120.87470920866618)=-10.600636217706475; dx=0.04054617062289626 evalInputDelta=-13.582097249688184
Armijo: th(13.430523245407352)=-11.41139558028627; dx=0.3707434224199023 evalInputDelta=-12.77133788710839
Armijo: th(1.3430523245407353)=-19.410349863015412; dx=3.2031696842326296 evalInputDelta=-4.772383604379247
New Minimum: -24.18273346739466 > -26.169402983059523
END: th(0.12209566586733957)=-26.169402983059523; dx=-13.640168871726786 evalInputDelta=1.9866695156648646
Fitness changed from -24.18273346739466 to -26.169402983059523
Iteration 14 complete. Error: -26.169402983059523 Total: 11.8145; Orientation: 0.2734; Line Search: 10.5402
Adding measurement 6deafcbb to history. Total: 4
Rejected: LBFGS Orientation magnitude: 1.834e+05, gradient 1.330e-03, dot -0.254; [14ab1b7b-ad31-48a9-b732-4070ec8505ba = 1.000/1.000e+00]
Orientation rejected. Popping history element from -26.169402983059523, -24.18273346739466, -3.1925383403938774, -2.179706050072849, -1.3594032098694149
Rejected: LBFGS Orientation magnitude: 4.956e+05, gradient 1.330e-03, dot -0.338; [14ab1b7b-ad31-48a9-b732-4070ec8505ba = 1.000/1.000e+00]
Orientation rejected. Popping history element from -26.169402983059523, -24.18273346739466, -3.1925383403938774, -2.179706050072849
LBFGS Accumulation History: 3 points
Removed measurement 6deafcbb to history. Total: 4
Removed measurement 27c03adc to history. Total: 3
th(0)=-26.169402983059523;dx=-1.7477954487546797E-6
New Minimum: -26.169402983059523 > -26.169404146570386
END: th(0.2630471380471382)=-26.169404146570386; dx=-1.2630954618437496E-6 evalInputDelta=1.1635108627672253E-6
Fitness changed from -26.169402983059523 to -26.169404146570386
Iteration 15 complete. Error: -26.169404146570386 Total: 3.0692; Orientation: 0.6042; Line Search: 1.6695
<a id="p-15"></a>Iteration 15
<a id="p-14"></a>
Final threshold in iteration 16: -26.169404146570386 (> -Infinity) after 71.485s (< 5400.000s)
Returns
-26.169404146570386