BasicOptimizer.scala:89 executed in 62.01 seconds (0.827 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 161930153800600
Reset training subject: 161931058911400
Adding measurement 100bdf5b to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-57.32616811527917;dx=-0.055925456726232434
New Minimum: -57.32616811527917 > -57.43005256396689
WOLFE (weak): th(2.154434690031884)=-57.43005256396689; dx=-0.05553382876935479 evalInputDelta=0.1038844486877224
New Minimum: -57.43005256396689 > -57.53418014333415
WOLFE (weak): th(4.308869380063768)=-57.53418014333415; dx=-0.05580498761317759 evalInputDelta=0.20801202805498065
New Minimum: -57.53418014333415 > -57.951218710222136
WOLFE (weak): th(12.926608140191302)=-57.951218710222136; dx=-0.05601451100269919 evalInputDelta=0.6250505949429694
New Minimum: -57.951218710222136 > -59.86382251488582
WOLFE (weak): th(51.70643256076521)=-59.86382251488582; dx=-0.05764793067429941 evalInputDelta=2.5376543996066516
New Minimum: -59.86382251488582 > -70.4003548353096
WOLFE (weak): th(258.53216280382605)=-70.4003548353096; dx=-0.058155105928881706 evalInputDelta=13.074186720030433
New Minimum: -70.4003548353096 > -115.96537737618068
END: th(1551.1929768229563)=-115.96537737618068; dx=-0.02711338566697668 evalInputDelta=58.63920926090152
Fitness changed from -57.32616811527917 to -115.96537737618068
Iteration 1 complete. Error: -115.96537737618068 Total: 8.6707; Orientation: 0.0148; Line Search: 6.1340
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 68305eca to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-115.96537737618068;dx=-0.08466900586355244
New Minimum: -115.96537737618068 > -209.68668032626678
END: th(3341.943960201201)=-209.68668032626678; dx=-0.014453402348930118 evalInputDelta=93.7213029500861
Fitness changed from -115.96537737618068 to -209.68668032626678
Iteration 2 complete. Error: -209.68668032626678 Total: 2.6166; Orientation: 0.0145; Line Search: 1.6538
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 3587d288 to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-209.68668032626678;dx=-0.07118278112689438
New Minimum: -209.68668032626678 > -325.60863682494664
END: th(7200.000000000001)=-325.60863682494664; dx=-0.0053459860866368145 evalInputDelta=115.92195649867986
Fitness changed from -209.68668032626678 to -325.60863682494664
Iteration 3 complete. Error: -325.60863682494664 Total: 2.6616; Orientation: 0.0149; Line Search: 1.7293
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 385e708e to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.042e+03, gradient 2.628e-01, dot -0.207; [2323558a-bbab-4e81-9dab-2cd5f6c2e805 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -325.60863682494664, -209.68668032626678, -115.96537737618068, -57.32616811527917
LBFGS Accumulation History: 3 points
Removed measurement 385e708e to history. Total: 3
th(0)=-325.60863682494664;dx=-0.06887737386899163
New Minimum: -325.60863682494664 > -468.8964437911638
END: th(15511.929768229566)=-468.8964437911638; dx=-6.404938500022262E-4 evalInputDelta=143.28780696621715
Fitness changed from -325.60863682494664 to -468.8964437911638
Iteration 4 complete. Error: -468.8964437911638 Total: 3.0948; Orientation: 0.2709; Line Search: 1.9086
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 1ef8fe3f to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.551e+03, gradient 2.625e-01, dot -0.170; [2323558a-bbab-4e81-9dab-2cd5f6c2e805 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -468.8964437911638, -209.68668032626678, -115.96537737618068, -57.32616811527917
LBFGS Accumulation History: 3 points
Removed measurement 1ef8fe3f to history. Total: 3
th(0)=-468.8964437911638;dx=-0.06836131395343045
New Minimum: -468.8964437911638 > -586.7151140033807
WOLF (strong): th(33419.43960201202)=-586.7151140033807; dx=0.0027156472294487884 evalInputDelta=117.81867021221689
New Minimum: -586.7151140033807 > -711.7636984983149
END: th(16709.71980100601)=-711.7636984983149; dx=-0.0011522961441473429 evalInputDelta=242.86725470715106
Fitness changed from -468.8964437911638 to -711.7636984983149
Iteration 5 complete. Error: -711.7636984983149 Total: 3.6470; Orientation: 0.2721; Line Search: 2.4951
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 1db545c6 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.903e+03, gradient 2.270e-01, dot -0.170; [2323558a-bbab-4e81-9dab-2cd5f6c2e805 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -711.7636984983149, -209.68668032626678, -115.96537737618068, -57.32616811527917
LBFGS Accumulation History: 3 points
Removed measurement 1db545c6 to history. Total: 3
th(0)=-711.7636984983149;dx=-0.051135331295274894
New Minimum: -711.7636984983149 > -854.8070323178208
WOLF (strong): th(36000.000000000015)=-854.8070323178208; dx=0.004083925474619775 evalInputDelta=143.04333381950596
New Minimum: -854.8070323178208 > -996.6712722859027
END: th(18000.000000000007)=-996.6712722859027; dx=-0.003969441552061531 evalInputDelta=284.9075737875878
Fitness changed from -711.7636984983149 to -996.6712722859027
Iteration 6 complete. Error: -996.6712722859027 Total: 3.7105; Orientation: 0.2746; Line Search: 2.5468
Adding measurement 518224d8 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.130e+04, gradient 2.140e-01, dot -0.226; [2323558a-bbab-4e81-9dab-2cd5f6c2e805 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -996.6712722859027, -209.68668032626678, -115.96537737618068, -57.32616811527917
LBFGS Accumulation History: 3 points
Removed measurement 518224d8 to history. Total: 3
th(0)=-996.6712722859027;dx=-0.04565595296096889
New Minimum: -996.6712722859027 > -1115.3419879122278
WOLF (strong): th(38779.824420573925)=-1115.3419879122278; dx=0.0014845185637698164 evalInputDelta=118.67071562632509
New Minimum: -1115.3419879122278 > -1250.0573986821564
END: th(19389.912210286962)=-1250.0573986821564; dx=-0.004570249033748561 evalInputDelta=253.38612639625376
Fitness changed from -996.6712722859027 to -1250.0573986821564
Iteration 7 complete. Error: -1250.0573986821564 Total: 3.8608; Orientation: 0.3847; Line Search: 2.6367
Adding measurement 4fe34df7 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.255e+04, gradient 2.129e-01, dot -0.266; [2323558a-bbab-4e81-9dab-2cd5f6c2e805 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1250.0573986821564, -209.68668032626678, -115.96537737618068, -57.32616811527917
LBFGS Accumulation History: 3 points
Removed measurement 4fe34df7 to history. Total: 3
th(0)=-1250.0573986821564;dx=-0.04520741909726059
New Minimum: -1250.0573986821564 > -1315.0640668803935
WOLF (strong): th(41774.29950251503)=-1315.0640668803935; dx=0.002940558683943803 evalInputDelta=65.0066681982371
New Minimum: -1315.0640668803935 > -1491.894768406941
END: th(20887.149751257515)=-1491.894768406941; dx=-0.0036920246729506205 evalInputDelta=241.8373697247846
Fitness changed from -1250.0573986821564 to -1491.894768406941
Iteration 8 complete. Error: -1491.894768406941 Total: 3.6635; Orientation: 0.2736; Line Search: 2.5746
Adding measurement 7798b611 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.566e+04, gradient 2.118e-01, dot -0.302; [2323558a-bbab-4e81-9dab-2cd5f6c2e805 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1491.894768406941, -209.68668032626678, -115.96537737618068, -57.32616811527917
LBFGS Accumulation History: 3 points
Removed measurement 7798b611 to history. Total: 3
th(0)=-1491.894768406941;dx=-0.04459493671178823
New Minimum: -1491.894768406941 > -1554.1738158906164
WOLF (strong): th(45000.00000000002)=-1554.1738158906164; dx=0.003419657187411733 evalInputDelta=62.27904748367541
New Minimum: -1554.1738158906164 > -1758.9472703864662
END: th(22500.00000000001)=-1758.9472703864662; dx=-0.002867750055302401 evalInputDelta=267.05250197952523
Fitness changed from -1491.894768406941 to -1758.9472703864662
Iteration 9 complete. Error: -1758.9472703864662 Total: 3.6264; Orientation: 0.2727; Line Search: 2.5445
Adding measurement 67aaf050 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.355e+05, gradient 2.111e-01, dot -0.323; [2323558a-bbab-4e81-9dab-2cd5f6c2e805 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1758.9472703864662, -209.68668032626678, -115.96537737618068, -57.32616811527917
LBFGS Accumulation History: 3 points
Removed measurement 67aaf050 to history. Total: 3
th(0)=-1758.9472703864662;dx=-0.044455284953689986
Armijo: th(48474.78052571741)=-1741.0164416225368; dx=0.004412225783874528 evalInputDelta=-17.930828763929412
New Minimum: -1758.9472703864662 > -2003.5675057257242
END: th(24237.390262858706)=-2003.5675057257242; dx=-0.00139442860305463 evalInputDelta=244.620235339258
Fitness changed from -1758.9472703864662 to -2003.5675057257242
Iteration 10 complete. Error: -2003.5675057257242 Total: 3.8122; Orientation: 0.2580; Line Search: 2.5732
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Adding measurement 4530eb6b to history. Total: 3
Rejected: LBFGS Orientation magnitude: 7.651e+06, gradient 2.144e-01, dot -0.329; [2323558a-bbab-4e81-9dab-2cd5f6c2e805 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -2003.5675057257242, -209.68668032626678, -115.96537737618068, -57.32616811527917
LBFGS Accumulation History: 3 points
Removed measurement 4530eb6b to history. Total: 3
th(0)=-2003.5675057257242;dx=-0.04584174408249965
Armijo: th(52217.874378143795)=-1961.4854065223121; dx=0.005878347778703305 evalInputDelta=-42.0820992034121
New Minimum: -2003.5675057257242 > -2278.639616086756
END: th(26108.937189071898)=-2278.639616086756; dx=-4.3410332085279033E-4 evalInputDelta=275.07211036103195
Fitness changed from -2003.5675057257242 to -2278.639616086756
Iteration 11 complete. Error: -2278.639616086756 Total: 3.9470; Orientation: 0.4555; Line Search: 2.5930
Adding measurement 226ac722 to history. Total: 3
Accepted: LBFGS Orientation magnitude: 1.189e+05, gradient 2.131e-01, dot 0.353; [2323558a-bbab-4e81-9dab-2cd5f6c2e805 = 1.000/1.000e+00]
Constructing line search parameters: LBFGS+Trust
th(0)=-2278.639616086756;dx=-8949.630608435838
Armijo: th(56250.00000000004)=-1994.7166465440648; dx=0.01420835015997768 evalInputDelta=-283.92296954269136
Armijo: th(28125.00000000002)=-1994.7142652520436; dx=0.028408946107257464 evalInputDelta=-283.9253508347126
Armijo: th(9375.000000000005)=-1994.9401620098783; dx=0.08513895154524848 evalInputDelta=-283.6994540768778
Armijo: th(2343.7500000000014)=-1995.1885249269053; dx=0.34108450793562617 evalInputDelta=-283.4510911598509
Armijo: th(468.7500000000003)=-1996.1084639868645; dx=1.703705966317767 evalInputDelta=-282.53115209989164
Armijo: th(78.12500000000004)=-2001.5064951181148; dx=10.201012691823268 evalInputDelta=-277.1331209686414
Armijo: th(11.160714285714292)=-2031.256876911436; dx=68.5055119508697 evalInputDelta=-247.38273917532024
Armijo: th(1.3950892857142865)=-2213.066989496703; dx=347.7812463708148 evalInputDelta=-65.572626590053
New Minimum: -2278.639616086756 > -2665.0078799133034
END: th(0.15500992063492072)=-2665.0078799133034; dx=-1550.0542488635078 evalInputDelta=386.36826382654726
Fitness changed from -2278.639616086756 to -2665.0078799133034
Iteration 12 complete. Error: -2665.0078799133034 Total: 9.7744; Orientation: 0.2651; Line Search: 8.6770
Adding measurement 7267b7bb to history. Total: 4
Rejected: LBFGS Orientation magnitude: 1.147e+05, gradient 2.131e-01, dot -0.189; [2323558a-bbab-4e81-9dab-2cd5f6c2e805 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -2665.0078799133034, -2278.639616086756, -209.68668032626678, -115.96537737618068, -57.32616811527917
Rejected: LBFGS Orientation magnitude: 3.049e+05, gradient 2.131e-01, dot -0.251; [2323558a-bbab-4e81-9dab-2cd5f6c2e805 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -2665.0078799133034, -2278.639616086756, -209.68668032626678, -115.96537737618068
LBFGS Accumulation History: 3 points
Removed measurement 7267b7bb to history. Total: 4
Removed measurement 226ac722 to history. Total: 3
th(0)=-2665.0078799133034;dx=-0.04529207728429594
New Minimum: -2665.0078799133034 > -2665.018386428131
END: th(0.33395875031496236)=-2665.018386428131; dx=-0.031207232363525322 evalInputDelta=0.01050651482773901
Fitness changed from -2665.0078799133034 to -2665.018386428131
Iteration 13 complete. Error: -2665.018386428131 Total: 3.0758; Orientation: 0.5910; Line Search: 1.7023
Adding measurement 63f5c810 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 8.174e+04, gradient 2.131e-01, dot -0.224; [2323558a-bbab-4e81-9dab-2cd5f6c2e805 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -2665.018386428131, -209.68668032626678, -115.96537737618068, -57.32616811527917
LBFGS Accumulation History: 3 points
Removed measurement 63f5c810 to history. Total: 3
th(0)=-2665.018386428131;dx=-0.04526187110408347
New Minimum: -2665.018386428131 > -2665.0408064418075
END: th(0.7194923167182512)=-2665.0408064418075; dx=-0.0313129574600334 evalInputDelta=0.02242001367631019
Fitness changed from -2665.018386428131 to -2665.0408064418075
Iteration 14 complete. Error: -2665.0408064418075 Total: 2.9587; Orientation: 0.2584; Line Search: 1.6636
Adding measurement 4c227ef8 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 8.223e+04, gradient 2.131e-01, dot -0.224; [2323558a-bbab-4e81-9dab-2cd5f6c2e805 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -2665.0408064418075, -209.68668032626678, -115.96537737618068, -57.32616811527917
LBFGS Accumulation History: 3 points
Removed measurement 4c227ef8 to history. Total: 3
th(0)=-2665.0408064418075;dx=-0.045328928480270814
New Minimum: -2665.0408064418075 > -2665.08921051435
END: th(1.5500992063492076)=-2665.08921051435; dx=-0.031305334060580906 evalInputDelta=0.048404072542325594
Fitness changed from -2665.0408064418075 to -2665.08921051435
Iteration 15 complete. Error: -2665.08921051435 Total: 2.7903; Orientation: 0.2691; Line Search: 1.7035
<a id="p-15"></a>Iteration 15
<a id="p-14"></a>
Final threshold in iteration 16: -2665.08921051435 (> -Infinity) after 62.010s (< 5400.000s)
Returns
-2665.08921051435