BasicOptimizer.scala:89 executed in 55.49 seconds (1.574 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 208355045000800
Reset training subject: 208355817760900
Adding measurement a40a2d1 to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-24.97495653819754;dx=-0.0328266603502076
New Minimum: -24.97495653819754 > -25.02838800593242
WOLFE (weak): th(2.154434690031884)=-25.02838800593242; dx=-0.03299557096226355 evalInputDelta=0.05343146773488172
New Minimum: -25.02838800593242 > -25.081964948768942
WOLFE (weak): th(4.308869380063768)=-25.081964948768942; dx=-0.0330617513012519 evalInputDelta=0.10700841057140309
New Minimum: -25.081964948768942 > -25.296900616809822
WOLFE (weak): th(12.926608140191302)=-25.296900616809822; dx=-0.03317748020135531 evalInputDelta=0.32194407861228314
New Minimum: -25.296900616809822 > -26.270070542369147
WOLFE (weak): th(51.70643256076521)=-26.270070542369147; dx=-0.033360683120143286 evalInputDelta=1.2951140041716087
New Minimum: -26.270070542369147 > -31.573873228631513
WOLFE (weak): th(258.53216280382605)=-31.573873228631513; dx=-0.03409460443400448 evalInputDelta=6.598916690433974
New Minimum: -31.573873228631513 > -61.25652030215197
END: th(1551.1929768229563)=-61.25652030215197; dx=-0.02258540816182337 evalInputDelta=36.28156376395443
Fitness changed from -24.97495653819754 to -61.25652030215197
Iteration 1 complete. Error: -61.25652030215197 Total: 7.7355; Orientation: 0.0157; Line Search: 5.2659
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 568a1a3f to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-61.25652030215197;dx=-0.03314476977636925
New Minimum: -61.25652030215197 > -108.87168050678102
END: th(3341.943960201201)=-108.87168050678102; dx=-0.011668180537056812 evalInputDelta=47.61516020462905
Fitness changed from -61.25652030215197 to -108.87168050678102
Iteration 2 complete. Error: -108.87168050678102 Total: 2.4547; Orientation: 0.0160; Line Search: 1.7202
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 377df2aa to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-108.87168050678102;dx=-0.03182059809049568
New Minimum: -108.87168050678102 > -160.90573472842212
END: th(7200.000000000001)=-160.90573472842212; dx=-0.006863457367447086 evalInputDelta=52.0340542216411
Fitness changed from -108.87168050678102 to -160.90573472842212
Iteration 3 complete. Error: -160.90573472842212 Total: 2.3003; Orientation: 0.0163; Line Search: 1.5796
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 6c1d9ee1 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.824e+03, gradient 2.157e-01, dot -0.302; [f083819b-cb93-45e0-b63c-2600930293af = 1.000/1.000e+00]
Orientation rejected. Popping history element from -160.90573472842212, -108.87168050678102, -61.25652030215197, -24.97495653819754
LBFGS Accumulation History: 3 points
Removed measurement 6c1d9ee1 to history. Total: 3
th(0)=-160.90573472842212;dx=-0.04642799713090909
New Minimum: -160.90573472842212 > -221.28746416025183
END: th(15511.929768229566)=-221.28746416025183; dx=-0.0043799705971020435 evalInputDelta=60.38172943182971
Fitness changed from -160.90573472842212 to -221.28746416025183
Iteration 4 complete. Error: -221.28746416025183 Total: 2.6303; Orientation: 0.3163; Line Search: 1.5344
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 4ab69830 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.178e+03, gradient 2.107e-01, dot -0.190; [f083819b-cb93-45e0-b63c-2600930293af = 1.000/1.000e+00]
Orientation rejected. Popping history element from -221.28746416025183, -108.87168050678102, -61.25652030215197, -24.97495653819754
LBFGS Accumulation History: 3 points
Removed measurement 4ab69830 to history. Total: 3
th(0)=-221.28746416025183;dx=-0.044135101541886664
New Minimum: -221.28746416025183 > -290.4264627404245
END: th(33419.43960201202)=-290.4264627404245; dx=-0.0017812672781270184 evalInputDelta=69.13899858017265
Fitness changed from -221.28746416025183 to -290.4264627404245
Iteration 5 complete. Error: -290.4264627404245 Total: 2.5483; Orientation: 0.3202; Line Search: 1.5185
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 79b27a9b to history. Total: 3
Rejected: LBFGS Orientation magnitude: 6.583e+03, gradient 2.126e-01, dot -0.121; [f083819b-cb93-45e0-b63c-2600930293af = 1.000/1.000e+00]
Orientation rejected. Popping history element from -290.4264627404245, -108.87168050678102, -61.25652030215197, -24.97495653819754
LBFGS Accumulation History: 3 points
Removed measurement 79b27a9b to history. Total: 3
th(0)=-290.4264627404245;dx=-0.045032455087983314
New Minimum: -290.4264627404245 > -336.9861467310722
END: th(72000.00000000003)=-336.9861467310722; dx=-1.3292751227681203E-4 evalInputDelta=46.55968399064773
Fitness changed from -290.4264627404245 to -336.9861467310722
Iteration 6 complete. Error: -336.9861467310722 Total: 3.0425; Orientation: 0.2980; Line Search: 1.5181
Adding measurement 40b3c2a0 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 8.857e+03, gradient 1.808e-01, dot -0.077; [f083819b-cb93-45e0-b63c-2600930293af = 1.000/1.000e+00]
Orientation rejected. Popping history element from -336.9861467310722, -108.87168050678102, -61.25652030215197, -24.97495653819754
LBFGS Accumulation History: 3 points
Removed measurement 40b3c2a0 to history. Total: 3
th(0)=-336.9861467310722;dx=-0.03261971138194816
New Minimum: -336.9861467310722 > -417.26225505312834
WOLF (strong): th(155119.2976822957)=-417.26225505312834; dx=1.9664575293934183E-4 evalInputDelta=80.27610832205613
New Minimum: -417.26225505312834 > -556.851649640779
END: th(77559.64884114785)=-556.851649640779; dx=-0.0016093995704993336 evalInputDelta=219.86550290970678
Fitness changed from -336.9861467310722 to -556.851649640779
Iteration 7 complete. Error: -556.851649640779 Total: 3.3106; Orientation: 0.2922; Line Search: 2.3248
Adding measurement 934c927 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.748e+04, gradient 1.643e-01, dot -0.082; [f083819b-cb93-45e0-b63c-2600930293af = 1.000/1.000e+00]
Orientation rejected. Popping history element from -556.851649640779, -108.87168050678102, -61.25652030215197, -24.97495653819754
LBFGS Accumulation History: 3 points
Removed measurement 934c927 to history. Total: 3
th(0)=-556.851649640779;dx=-0.026920623333582003
New Minimum: -556.851649640779 > -569.0404655997406
WOLF (strong): th(167097.19801006012)=-569.0404655997406; dx=4.0936774045934954E-4 evalInputDelta=12.18881595896164
New Minimum: -569.0404655997406 > -816.2332873560117
END: th(83548.59900503006)=-816.2332873560117; dx=-0.0018663938974036293 evalInputDelta=259.3816377152327
Fitness changed from -556.851649640779 to -816.2332873560117
Iteration 8 complete. Error: -816.2332873560117 Total: 3.5443; Orientation: 0.3040; Line Search: 2.5109
Adding measurement 5c4ee881 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.213e+04, gradient 1.666e-01, dot -0.098; [f083819b-cb93-45e0-b63c-2600930293af = 1.000/1.000e+00]
Orientation rejected. Popping history element from -816.2332873560117, -108.87168050678102, -61.25652030215197, -24.97495653819754
LBFGS Accumulation History: 3 points
Removed measurement 5c4ee881 to history. Total: 3
th(0)=-816.2332873560117;dx=-0.027688636628103447
Armijo: th(180000.0000000001)=-672.3548885320382; dx=6.429479354976091E-4 evalInputDelta=-143.8783988239735
New Minimum: -816.2332873560117 > -1019.9887538615026
END: th(90000.00000000004)=-1019.9887538615026; dx=-0.0014203076486481838 evalInputDelta=203.75546650549086
Fitness changed from -816.2332873560117 to -1019.9887538615026
Iteration 9 complete. Error: -1019.9887538615026 Total: 3.2421; Orientation: 0.2808; Line Search: 2.2853
Adding measurement 574f398a to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.257e+05, gradient 1.635e-01, dot -0.103; [f083819b-cb93-45e0-b63c-2600930293af = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1019.9887538615026, -108.87168050678102, -61.25652030215197, -24.97495653819754
LBFGS Accumulation History: 3 points
Removed measurement 574f398a to history. Total: 3
th(0)=-1019.9887538615026;dx=-0.026704084735752073
Armijo: th(193899.12210286965)=-790.9686085221572; dx=9.851340755264542E-4 evalInputDelta=-229.0201453393454
New Minimum: -1019.9887538615026 > -1226.3056698493963
END: th(96949.56105143482)=-1226.3056698493963; dx=-0.0014030441847931352 evalInputDelta=206.3169159878937
Fitness changed from -1019.9887538615026 to -1226.3056698493963
Iteration 10 complete. Error: -1226.3056698493963 Total: 3.5676; Orientation: 0.2907; Line Search: 2.5170
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Adding measurement 1635648e to history. Total: 3
Rejected: LBFGS Orientation magnitude: 6.697e+06, gradient 1.644e-01, dot -0.107; [f083819b-cb93-45e0-b63c-2600930293af = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1226.3056698493963, -108.87168050678102, -61.25652030215197, -24.97495653819754
LBFGS Accumulation History: 3 points
Removed measurement 1635648e to history. Total: 3
th(0)=-1226.3056698493963;dx=-0.026974783064398378
Armijo: th(208871.49751257518)=-886.616338667544; dx=0.0011888094107953705 evalInputDelta=-339.68933118185225
New Minimum: -1226.3056698493963 > -1374.9092787769612
END: th(104435.74875628759)=-1374.9092787769612; dx=-7.6765631772148E-4 evalInputDelta=148.6036089275649
Fitness changed from -1226.3056698493963 to -1374.9092787769612
Iteration 11 complete. Error: -1374.9092787769612 Total: 3.2359; Orientation: 0.2753; Line Search: 2.2197
Adding measurement 7cae7566 to history. Total: 3
Accepted: LBFGS Orientation magnitude: 1.109e+05, gradient 1.638e-01, dot 0.116; [f083819b-cb93-45e0-b63c-2600930293af = 1.000/1.000e+00]
Constructing line search parameters: LBFGS+Trust
th(0)=-1374.9092787769612;dx=-2095.2080865316393
Armijo: th(225000.00000000015)=-728.3431735967927; dx=0.0030082690356842898 evalInputDelta=-646.5661051801685
Armijo: th(112500.00000000007)=-728.3475208291013; dx=0.006004547214701504 evalInputDelta=-646.5617579478599
Armijo: th(37500.00000000002)=-728.4208922123114; dx=0.017968654537387028 evalInputDelta=-646.4883865646498
Armijo: th(9375.000000000005)=-728.4940997073725; dx=0.07210468276251916 evalInputDelta=-646.4151790695887
Armijo: th(1875.0000000000011)=-728.4024472393442; dx=0.36026051383627383 evalInputDelta=-646.506831537617
Armijo: th(312.50000000000017)=-729.2836596902328; dx=2.1484417212134224 evalInputDelta=-645.6256190867284
Armijo: th(44.64285714285717)=-736.4301031879118; dx=14.88147150095209 evalInputDelta=-638.4791755890494
Armijo: th(5.580357142857146)=-798.777627149281; dx=107.05750583021754 evalInputDelta=-576.1316516276802
Armijo: th(0.6200396825396829)=-1205.4522971320982; dx=340.3023100475509 evalInputDelta=-169.45698164486294
New Minimum: -1374.9092787769612 > -1411.8281278755592
END: th(0.06200396825396829)=-1411.8281278755592; dx=-367.8258690601371 evalInputDelta=36.918849098598
Fitness changed from -1374.9092787769612 to -1411.8281278755592
Iteration 12 complete. Error: -1411.8281278755592 Total: 9.5126; Orientation: 0.2850; Line Search: 8.5069
Adding measurement 38742a55 to history. Total: 4
Rejected: LBFGS Orientation magnitude: 4.539e+05, gradient 1.574e-01, dot -0.083; [f083819b-cb93-45e0-b63c-2600930293af = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1411.8281278755592, -1374.9092787769612, -108.87168050678102, -61.25652030215197, -24.97495653819754
Rejected: LBFGS Orientation magnitude: 6.657e+05, gradient 1.574e-01, dot -0.088; [f083819b-cb93-45e0-b63c-2600930293af = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1411.8281278755592, -1374.9092787769612, -108.87168050678102, -61.25652030215197
LBFGS Accumulation History: 3 points
Removed measurement 38742a55 to history. Total: 4
Removed measurement 7cae7566 to history. Total: 3
th(0)=-1411.8281278755592;dx=-0.02474826196369767
New Minimum: -1411.8281278755592 > -1411.829733326219
END: th(0.13358350012598494)=-1411.829733326219; dx=-0.019067870478223312 evalInputDelta=0.0016054506597811269
Fitness changed from -1411.8281278755592 to -1411.829733326219
Iteration 13 complete. Error: -1411.829733326219 Total: 3.0661; Orientation: 0.8270; Line Search: 1.5498
Adding measurement 7df8df84 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.547e+04, gradient 1.574e-01, dot -0.060; [f083819b-cb93-45e0-b63c-2600930293af = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1411.829733326219, -108.87168050678102, -61.25652030215197, -24.97495653819754
LBFGS Accumulation History: 3 points
Removed measurement 7df8df84 to history. Total: 3
th(0)=-1411.829733326219;dx=-0.02473332338924442
New Minimum: -1411.829733326219 > -1411.8332343878478
END: th(0.28779692668730045)=-1411.8332343878478; dx=-0.01907138773047143 evalInputDelta=0.003501061628867319
Fitness changed from -1411.829733326219 to -1411.8332343878478
Iteration 14 complete. Error: -1411.8332343878478 Total: 2.5164; Orientation: 0.3121; Line Search: 1.5568
Adding measurement 1eb3ed4d to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.876e+04, gradient 1.575e-01, dot -0.062; [f083819b-cb93-45e0-b63c-2600930293af = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1411.8332343878478, -108.87168050678102, -61.25652030215197, -24.97495653819754
LBFGS Accumulation History: 3 points
Removed measurement 1eb3ed4d to history. Total: 3
th(0)=-1411.8332343878478;dx=-0.024764872001810444
New Minimum: -1411.8332343878478 > -1411.8409326096844
END: th(0.6200396825396829)=-1411.8409326096844; dx=-0.01906529678544442 evalInputDelta=0.00769822183656288
Fitness changed from -1411.8332343878478 to -1411.8409326096844
Iteration 15 complete. Error: -1411.8409326096844 Total: 2.7574; Orientation: 0.2896; Line Search: 1.7492
<a id="p-15"></a>Iteration 15
<a id="p-14"></a>
Final threshold in iteration 16: -1411.8409326096844 (> -Infinity) after 55.488s (< 5400.000s)
Returns
-1411.8409326096844