Numeric instability on linux when using lbfgs, minimization does not converge.

[racinmat]

On linux server, the style transfer is not working when using lbfgs, the output is following.

RUNNING THE L-BFGS-B CODE

           * * *

Machine precision = 2.220D-16
 N =       786432     M =           10
 This problem is unconstrained.

At X0         0 variables are exactly at the bounds

At iterate    0    f=  5.45129D+11    |proj g|=  4.00000D-03

           * * *

Tit   = total number of iterations
Tnf   = total number of function evaluations
Tnint = total number of segments explored during Cauchy searches
Skip  = number of BFGS updates skipped
Nact  = number of active bounds at final generalized Cauchy point
Projg = norm of the final projected gradient
F     = final function value

           * * *

   N    Tit     Tnf  Tnint  Skip  Nact     Projg        F
*****      1      3      1     0     0         NaN   5.451D+11
  F =   545128906752.00000

CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH

The exactly same code on windows machine is running just fine, on both machines the input data are the same, and when I checked inside the scipy library, specifically https://github.com/scipy/scipy/blob/v1.4.1/scipy/optimize/lbfgsb.py#L49-L208.
all arrays seemed to have same data type, although there were both float32 and float64 types.

I'm using this library in following conda environment:

name: tensorflow_1_14
channels:
  - defaults
dependencies:
  - _libgcc_mutex=0.1=main
  - _tflow_select=2.3.0=mkl
  - absl-py=0.9.0=py36_0
  - astor=0.8.0=py36_0
  - blas=1.0=mkl
  - c-ares=1.15.0=h7b6447c_1001
  - ca-certificates=2020.1.1=0
  - certifi=2020.4.5.1=py36_0
  - gast=0.3.3=py_0
  - google-pasta=0.2.0=py_0
  - grpcio=1.27.2=py36hf8bcb03_0
  - h5py=2.10.0=py36h7918eee_0
  - hdf5=1.10.4=hb1b8bf9_0
  - intel-openmp=2020.0=166
  - joblib=0.15.1=py_0
  - keras-applications=1.0.8=py_0
  - keras-preprocessing=1.1.0=py_1
  - ld_impl_linux-64=2.33.1=h53a641e_7
  - libedit=3.1.20181209=hc058e9b_0
  - libffi=3.2.1=hd88cf55_4
  - libgcc-ng=9.1.0=hdf63c60_0
  - libgfortran-ng=7.3.0=hdf63c60_0
  - libprotobuf=3.11.4=hd408876_0
  - libstdcxx-ng=9.1.0=hdf63c60_0
  - markdown=3.1.1=py36_0
  - mkl=2020.0=166
  - mkl-service=2.3.0=py36he904b0f_0
  - mkl_fft=1.0.15=py36ha843d7b_0
  - mkl_random=1.1.0=py36hd6b4f25_0
  - ncurses=6.2=he6710b0_1
  - numpy-base=1.16.6=py36hde5b4d6_0
  - openssl=1.1.1g=h7b6447c_0
  - pip=20.0.2=py36_1
  - protobuf=3.11.4=py36he6710b0_0
  - python=3.6.10=hcf32534_1
  - readline=8.0=h7b6447c_0
  - scikit-learn=0.22.1=py36hd81dba3_0
  - scipy=1.4.1=py36h0b6359f_0
  - setuptools=46.1.3=py36_0
  - six=1.14.0=py36_0
  - sqlite=3.31.1=h62c20be_1
  - tensorboard=1.14.0=py36hf484d3e_0
  - tensorflow=1.14.0=mkl_py36h2526735_0
  - tensorflow-base=1.14.0=mkl_py36h7ce6ba3_0
  - tensorflow-estimator=1.14.0=py_0
  - termcolor=1.1.0=py36_1
  - tk=8.6.8=hbc83047_0
  - werkzeug=1.0.1=py_0
  - wheel=0.34.2=py36_0
  - wrapt=1.12.1=py36h7b6447c_1
  - xz=5.2.5=h7b6447c_0
  - zlib=1.2.11=h7b6447c_3
  - pip:
    - chardet==3.0.4
    - click==7.1.2
    - decorator==4.4.2
    - idna==2.9
    - imageio==2.8.0
    - imageio-ffmpeg==0.4.1
    - moviepy==1.0.2
    - numpy==1.16.6
    - opencv-contrib-python-headless==4.2.0.34
    - opencv-python-headless==4.2.0.34
    - pillow==7.1.2
    - proglog==0.1.9
    - progressbar2==3.51.3
    - python-utils==2.4.0
    - requests==2.23.0
    - tqdm==4.45.0
    - urllib3==1.25.9

any help would be appreciated, I'm a bit lost in what can I do about this.
Is this some bad version of library, too new/too old? Or something else?

I assume it might be same issue as #99 .