BasicOptimizer.scala:89 executed in 297.90 seconds (6.508 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 589251039844800
Reset training subject: 589257113365300
Corrupt weights measurement
LBFGS Accumulation History: 0 points
Constructing line search parameters: GD+Trust
New Minimum: -35.681399242277415 > -35.72270202478261
th(0)=-35.72270202478261;dx=-1.910740901946046E-6
New Minimum: -35.72270202478261 > -35.72270625331189
END: th(2.154434690031884)=-35.72270625331189; dx=-1.7037704981860877E-6 evalInputDelta=4.228529277838788E-6
Fitness changed from -35.681399242277415 to -35.72270625331189
Iteration 1 complete. Error: -35.72270625331189 Total: 30.3624; Orientation: 0.1019; Line Search: 12.3607
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 2ed10bdf to history. Total: 0
LBFGS Accumulation History: 1 points
th(0)=-35.72270625331189;dx=-1.7188211448067745E-6
New Minimum: -35.72270625331189 > -35.7227168246329
WOLFE (weak): th(4.641588833612779)=-35.7227168246329; dx=-1.718886783096095E-6 evalInputDelta=1.0571321006125345E-5
New Minimum: -35.7227168246329 > -35.72272528168745
WOLFE (weak): th(9.283177667225559)=-35.72272528168745; dx=-1.7183699066572843E-6 evalInputDelta=1.902837556144732E-5
New Minimum: -35.72272528168745 > -35.72276122414696
WOLFE (weak): th(27.849533001676676)=-35.72276122414696; dx=-1.7172200135776734E-6 evalInputDelta=5.497083506611489E-5
New Minimum: -35.72276122414696 > -35.72291767913761
WOLFE (weak): th(111.3981320067067)=-35.72291767913761; dx=-1.7159061648377692E-6 evalInputDelta=2.114258257179813E-4
New Minimum: -35.72291767913761 > -35.72376125572555
WOLFE (weak): th(556.9906600335335)=-35.72376125572555; dx=-1.7160533280139613E-6 evalInputDelta=0.0010550024136577463
New Minimum: -35.72376125572555 > -35.72903580051416
WOLFE (weak): th(3341.9439602012008)=-35.72903580051416; dx=-1.7149498735598785E-6 evalInputDelta=0.0063295472022701915
New Minimum: -35.72903580051416 > -35.76698111471677
WOLFE (weak): th(23393.607721408407)=-35.76698111471677; dx=-1.7139994727756914E-6 evalInputDelta=0.04427486140487957
New Minimum: -35.76698111471677 > -36.07516807234101
WOLFE (weak): th(187148.86177126726)=-36.07516807234101; dx=-1.6968672905022405E-6 evalInputDelta=0.35246181902911644
New Minimum: -36.07516807234101 > -38.71279215716003
END: th(1684339.7559414054)=-38.71279215716003; dx=-1.4908862982657746E-6 evalInputDelta=2.99008590384814
Fitness changed from -35.72270625331189 to -38.71279215716003
Iteration 2 complete. Error: -38.71279215716003 Total: 68.0801; Orientation: 0.1429; Line Search: 60.4492
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 3354d853 to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-38.71279215716003;dx=-1.4475734183199157E-6
New Minimum: -38.71279215716003 > -43.553781023448074
END: th(3628800.0000000005)=-43.553781023448074; dx=-1.0139922123100844E-6 evalInputDelta=4.840988866288043
Fitness changed from -38.71279215716003 to -43.553781023448074
Iteration 3 complete. Error: -43.553781023448074 Total: 19.6348; Orientation: 0.1281; Line Search: 12.3191
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 749cf836 to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-43.553781023448074;dx=-9.866246704562688E-7
New Minimum: -43.553781023448074 > -50.17346630877401
END: th(7818012.603187701)=-50.17346630877401; dx=-5.865922509154939E-7 evalInputDelta=6.619685285325936
Fitness changed from -43.553781023448074 to -50.17346630877401
Iteration 4 complete. Error: -50.17346630877401 Total: 20.0873; Orientation: 0.1304; Line Search: 12.6015
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 56a466e0 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.961e+04, gradient 8.764e-04, dot -0.732; [73232db8-2ec8-4eda-9260-2d68536b0a60 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -50.17346630877401, -43.553781023448074, -38.71279215716003, -35.72270625331189
LBFGS Accumulation History: 3 points
Removed measurement 56a466e0 to history. Total: 3
th(0)=-50.17346630877401;dx=-7.680977792849253E-7
New Minimum: -50.17346630877401 > -57.88999603401668
END: th(1.6843397559414055E7)=-57.88999603401668; dx=-2.2961462066455696E-7 evalInputDelta=7.7165297252426726
Fitness changed from -50.17346630877401 to -57.88999603401668
Iteration 5 complete. Error: -57.88999603401668 Total: 22.2582; Orientation: 2.6336; Line Search: 12.5080
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 5de83a64 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.004e+04, gradient 9.823e-04, dot -0.492; [73232db8-2ec8-4eda-9260-2d68536b0a60 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -57.88999603401668, -43.553781023448074, -38.71279215716003, -35.72270625331189
LBFGS Accumulation History: 3 points
Removed measurement 5de83a64 to history. Total: 3
th(0)=-57.88999603401668;dx=-9.624567311119485E-7
New Minimum: -57.88999603401668 > -59.9684669316029
WOLF (strong): th(3.628800000000001E7)=-59.9684669316029; dx=1.1516943781979813E-7 evalInputDelta=2.0784708975862145
New Minimum: -59.9684669316029 > -63.073761701174035
WOLF (strong): th(1.8144000000000004E7)=-63.073761701174035; dx=1.1197947578910172E-7 evalInputDelta=5.183765667157353
END: th(6048000.000000001)=-62.5379916987233; dx=-4.195869832035066E-7 evalInputDelta=4.64799566470662
Fitness changed from -57.88999603401668 to -63.073761701174035
Iteration 6 complete. Error: -63.073761701174035 Total: 34.3752; Orientation: 2.4770; Line Search: 24.7249
Adding measurement 1f5d756a to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.389e+04, gradient 1.069e-03, dot -0.372; [73232db8-2ec8-4eda-9260-2d68536b0a60 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -63.073761701174035, -43.553781023448074, -38.71279215716003, -35.72270625331189
LBFGS Accumulation History: 3 points
Removed measurement 1f5d756a to history. Total: 3
th(0)=-63.073761701174035;dx=-1.1413676246708013E-6
New Minimum: -63.073761701174035 > -71.35243007537909
WOLF (strong): th(1.3030021005312836E7)=-71.35243007537909; dx=2.3279321404060636E-8 evalInputDelta=8.278668374205054
END: th(6515010.502656418)=-69.52773936810728; dx=-5.966005328244867E-7 evalInputDelta=6.453977666933241
Fitness changed from -63.073761701174035 to -71.35243007537909
Iteration 7 complete. Error: -71.35243007537909 Total: 26.6749; Orientation: 2.4368; Line Search: 18.2319
Adding measurement 55a6f501 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.203e+04, gradient 8.180e-04, dot -0.505; [73232db8-2ec8-4eda-9260-2d68536b0a60 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -71.35243007537909, -43.553781023448074, -38.71279215716003, -35.72270625331189
LBFGS Accumulation History: 3 points
Removed measurement 55a6f501 to history. Total: 3
th(0)=-71.35243007537909;dx=-6.688077696874477E-7
New Minimum: -71.35243007537909 > -76.10919511296052
END: th(1.4036164632845048E7)=-76.10919511296052; dx=-1.5157378555956172E-8 evalInputDelta=4.756765037581431
Fitness changed from -71.35243007537909 to -76.10919511296052
Iteration 8 complete. Error: -76.10919511296052 Total: 20.9390; Orientation: 2.3796; Line Search: 12.5847
Adding measurement 71d52687 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.974e+04, gradient 8.021e-04, dot -0.491; [73232db8-2ec8-4eda-9260-2d68536b0a60 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -76.10919511296052, -43.553781023448074, -38.71279215716003, -35.72270625331189
LBFGS Accumulation History: 3 points
Removed measurement 71d52687 to history. Total: 3
th(0)=-76.10919511296052;dx=-6.431525671615288E-7
New Minimum: -76.10919511296052 > -77.40864419475388
WOLF (strong): th(3.024000000000001E7)=-77.40864419475388; dx=1.9121522762666355E-7 evalInputDelta=1.2994490817933553
New Minimum: -77.40864419475388 > -80.26325343995991
WOLF (strong): th(1.5120000000000006E7)=-80.26325343995991; dx=5.309803707955977E-8 evalInputDelta=4.15405832699939
END: th(5040000.000000002)=-78.96476624126505; dx=-3.8193107416483606E-7 evalInputDelta=2.8555711283045326
Fitness changed from -76.10919511296052 to -80.26325343995991
Iteration 9 complete. Error: -80.26325343995991 Total: 33.4303; Orientation: 2.4920; Line Search: 25.0291
Adding measurement 1d68621c to history. Total: 3
Rejected: LBFGS Orientation magnitude: 6.507e+04, gradient 8.133e-04, dot -0.457; [73232db8-2ec8-4eda-9260-2d68536b0a60 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -80.26325343995991, -43.553781023448074, -38.71279215716003, -35.72270625331189
LBFGS Accumulation History: 3 points
Removed measurement 1d68621c to history. Total: 3
th(0)=-80.26325343995991;dx=-6.610354652923004E-7
New Minimum: -80.26325343995991 > -84.68602047200623
END: th(1.0858350837760698E7)=-84.68602047200623; dx=-7.38918138728534E-8 evalInputDelta=4.422767032046323
Fitness changed from -80.26325343995991 to -84.68602047200623
Iteration 10 complete. Error: -84.68602047200623 Total: 20.9583; Orientation: 2.4710; Line Search: 12.4944
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Final threshold in iteration 11: -84.68602047200623 (> -Infinity) after 297.901s (< 5400.000s)
Returns
-84.68602047200623