大家好，我剛學(xué)神經(jīng)網(wǎng)路不久。我想到一個regression實驗，想測試神經(jīng)網(wǎng)路能不能學(xué)到這個特例。這是一個簡單的試驗，training data是隨機(jī)產(chǎn)生的10,000筆資料，每筆資料特征向量維度為10x1，label為特征向量第1個元素值。

from numpy.random import RandomState
rdm=RandomState(1)
data_size=10000
xdim=10
X=rdm.rand(data_size,xdim)
Y = [x1[0] for x1 in X]

我用一個一層的網(wǎng)路來訓(xùn)練，預(yù)期輸出是Weights=[1,0,0,0,0,0,0,0,0,0]，bias=0。

我分別寫了tensorflow跟keras兩種版本。奇怪的是keras可以得到正確結(jié)果，tensorflow的訓(xùn)練卻無法收斂。而且兩個版本的loss差異很大。

Tensorflow版本:

import tensorflow as tf
x=tf.placeholder(tf.float64,shape=(None,xdim))
y=tf.placeholder(tf.float64,shape=(None))
Weights = tf.Variable(tf.random_normal([xdim, 1],dtype=tf.float64))
biases = tf.Variable(0.1,dtype=tf.float64)
y_predict = tf.matmul(x, Weights) + biases
loss = tf.reduce_mean(tf.square(y_predict - y))
optimizer = tf.train.GradientDescentOptimizer(0.01).minimize(loss)

batch_size=100
with tf.Session() as sess:
    sess.run(tf.global_variables_initializer())
    for i in range(10001):
        start = i * batch_size % data_size
        end = min(start + batch_size,data_size)
        sess.run(optimizer,feed_dict={x:X[start:end],y:Y[start:end]})
        if i % 1000 == 0:
            ypred,training_loss= sess.run([y_predict,loss],feed_dict={x:X,y:Y})
            print("Epoch %d: loss=%g"%(i,training_loss))

Tensorflow版本的輸出:

Epoch 0: loss=1.0679
Epoch 1000: loss=0.11685
Epoch 2000: loss=0.0842979
Epoch 3000: loss=0.0827121
Epoch 4000: loss=0.0824983
Epoch 5000: loss=0.0824296
Epoch 6000: loss=0.0824021
Epoch 7000: loss=0.0823903
Epoch 8000: loss=0.0823851
Epoch 9000: loss=0.0823826
Epoch 10000: loss=0.0823814

Keras版本:

from keras.models import Sequential
from keras.layers import Dense
import numpy as np

model = Sequential()
model.add(Dense(units=1, input_dim=xdim)) 
model.compile(loss='mse', optimizer='sgd')

batch_size=100
for i in range(10001):
    start = i * batch_size % data_size
    end = min(start + batch_size,data_size)
    cost = model.train_on_batch(X[start:end], np.array(Y[start:end]))
    if i % 1000 == 0:
        print("Epoch %d: loss=%g"%(i,cost))

Keras版本的輸出:

Epoch 0: loss=0.261707
Epoch 1000: loss=0.00811771
Epoch 2000: loss=0.000325865
Epoch 3000: loss=2.21623e-05
Epoch 4000: loss=4.63907e-06
Epoch 5000: loss=1.66684e-06
Epoch 6000: loss=6.55329e-07
Epoch 7000: loss=2.61024e-07
Epoch 8000: loss=1.04213e-07
Epoch 9000: loss=4.16416e-08
Epoch 10000: loss=1.66369e-08

我認(rèn)為兩段代碼應(yīng)該是等效的，差別是我不知道keras內(nèi)部如何設(shè)定learning rate。為什么keras可以得到正確結(jié)果，tensorflow卻不行呢？請問我哪邊弄錯呢？