deep learning Andrew學習筆記

標簽：學習

Freezing one weight cause the same result with unfreezing（Deep learning書的一個練習）

在神經網絡訓啦中，凍結某權重，總的誤差依然會降為0的現象。

# 用燈的組合與能否通信數據來做
import numpy as np
import matplotlib.pyplot as plt
streetlights = np.array([
    [1,0,1],
    [0,1,1],
    [0,0,1],
    [1,1,1],
    [0,1,1],
    [1,0,1]
])
walk_vs_stop = np.array([0,1,0,1,1,0])

#定義相關函數
alpha = 0.1
def predict(X,W):
    return np.dot(X,W)

#訓練
weight = np.array([0.5,0.48,-0.7]) #為了好看，weight設置個固定值
error = []
w = [[0.5,0.48,-0.7]]
for i in range(6):
    pre = predict(streetlights[i],weight)
    err = (pre - walk_vs_stop[i])**2
    delta = pre - walk_vs_stop[i]
    weight_delta = streetlights[i] * delta * alpha
    weight = weight - weight_delta
    if i != 5:
        w.append(weight)
    error.append(np.array(err))
print(error)
print(w)
fig, ax = plt.subplots()
ax.scatter(range(6),error)
plt.xlabel('run_times')
plt.ylabel('error')
plt.show()

[array(0.04), array(1.44), array(0.3136), array(0.147456), array(0.68425984), array(0.03080727)]
[[0.5, 0.48, -0.7], array([ 0.52,  0.48, -0.68]), array([ 0.52,  0.6 , -0.56]), array([ 0.52 ,  0.6  , -0.504]), array([ 0.5584,  0.6384, -0.4656]), array([ 0.5584 ,  0.72112, -0.38288])]

在這里插入圖片描述

#同樣的方法訓練，只是每次不變最后一個
weight = np.array([0.5,0.48,-0.7]) #為了好看，weight設置個固定值
error = []
w = [[0.5,0.48,-0.7]]
for i in range(6):
    pre = predict(streetlights[i],weight)
    err = (pre - walk_vs_stop[i])**2
    delta = pre - walk_vs_stop[i]
    weight_delta = streetlights[i] * delta * alpha
    weight_delta[2] = 0 #新加的語句
    weight = weight - weight_delta
    if i != 5:
        w.append(weight)
    error.append(np.array(err))
print(error)
print(w)
fig, ax = plt.subplots()
ax.scatter(range(6),error)
plt.xlabel('run_times')
plt.ylabel('error')
plt.show()

[array(0.04), array(1.4884), array(0.49), array(0.334084), array(1.08201604), array(0.01493284)]
[[0.5, 0.48, -0.7], array([ 0.52,  0.48, -0.7 ]), array([ 0.52 ,  0.602, -0.7  ]), array([ 0.52 ,  0.602, -0.7  ]), array([ 0.5778,  0.6598, -0.7   ]), array([ 0.5778 ,  0.76382, -0.7    ])]

在這里插入圖片描述

看到這兩次輸出的weight結果差別不大，說明第三個weight確實不怎么影響結果，換個weight試試，還是error還是降低到了0……，控制兩個呢

#同樣的方法訓練，只是每次不變最后一個
weight = np.array([0.5,0.48,-0.7]) #為了好看，weight設置個固定值
error = []
w = [[0.5,0.48,-0.7]]
for i in range(6):
    pre = predict(streetlights[i],weight)
    err = (pre - walk_vs_stop[i])**2
    delta = pre - walk_vs_stop[i]
    weight_delta = streetlights[i] * delta * alpha
    weight_delta[1] = 0 #新加的語句
    weight = weight - weight_delta
    if i != 5:
        w.append(weight)
    error.append(np.array(err))
print(error)
print(w)
fig, ax = plt.subplots()
ax.scatter(range(6),error)
plt.xlabel('run_times')
plt.ylabel('error')
plt.show()

[array(0.04), array(1.44), array(0.3136), array(0.254016), array(0.94789696), array(0.04586451)]
[[0.5, 0.48, -0.7], array([ 0.52,  0.48, -0.68]), array([ 0.52,  0.48, -0.56]), array([ 0.52 ,  0.48 , -0.504]), array([ 0.5704,  0.48  , -0.4536]), array([ 0.5704 ,  0.48   , -0.35624])]

在這里插入圖片描述

控制兩個，也一樣的

#同樣的方法訓練，只是每次不變最后一個
weight = np.array([0.5,0.48,-0.7]) #為了好看，weight設置個固定值
error = []
w = [[0.5,0.48,-0.7]]
for i in range(6):
    pre = predict(streetlights[i],weight)
    err = (pre - walk_vs_stop[i])**2
    delta = pre - walk_vs_stop[i]
    weight_delta = streetlights[i] * delta * alpha
    weight_delta[0] = 0 #新加的語句
    weight_delta[1] = 0 #新加的語句
    weight = weight - weight_delta
    if i != 5:
        w.append(weight)
    error.append(np.array(err))
print(error)
print(w)
fig, ax = plt.subplots()
ax.scatter(range(6),error)
plt.xlabel('run_times')
plt.ylabel('error')
plt.show()

[array(0.04), array(1.44), array(0.3136), array(0.274576), array(0.94400656), array(0.02118771)]
[[0.5, 0.48, -0.7], array([ 0.5 ,  0.48, -0.68]), array([ 0.5 ,  0.48, -0.56]), array([ 0.5  ,  0.48 , -0.504]), array([ 0.5   ,  0.48  , -0.4516]), array([ 0.5    ,  0.48   , -0.35444])]

在這里插入圖片描述

本文鏈接：https://blog.csdn.net/zjh12312311/article/details/108652208

智能推薦

deep learning學習筆記---MemN2N

摘要： MemN2N簡單介紹 MemN2N 1. Summary MemN2N is a generalization of RNN 1) The sentence in MemN2N is equivalent to the word in RNN; 2. Kernel Code Build Model Build Memory 3. Reference Blog: Memory-ne...

《deep learning》學習筆記（6）——深度前饋網絡

http://blog.csdn.net/u011239443/article/details/78148087 6.1 實例：學習 XOR 通過學習一個表示來解決 XOR 問題。圖上的粗體數字標明了學得的函數必須在每個點輸出的值。(左) 直接應用于原始輸入的線性模型不能實現 XOR 函數。當 x 1 = 0 時，模型的輸出必須隨著 x 2 的增大而增大。當 x 1 = 1 時，模型的輸出必須隨著...

《deep learning》學習筆記（2）——線性代數

http://blog.csdn.net/u011239443/article/details/77942575 2.1 標量、向量、矩陣和張量在numpy中，可以用以下方式生成各種維度的張量： 2.2 矩陣和向量相乘元素對應運算(Element-wise Operation)：針對形狀相同張量的運算統稱，包括元素對應乘積、相加等，即對兩個張量相同位置的元素進行加減乘除等運算。元素對應乘積（e...

機器學習筆記之Neural Networks (Deep Learning) MLPs

機器學習筆記之Neural Networks (Deep Learning) MLPs Here only discussed some relatively simple methods, namely multiplayer perceptrons for classifictaion and regression MLPs are feed-forward neural networks. ...

H2O學習筆記（四）——Deep Learning 例子

這里寫圖片描述##MNIST Digit Classification MNIST一個比較出名的數據集，包括60000張訓練圖片和10000張測試圖片，每張圖片是一個手寫數字，包括282像素值的手寫識別數據,掃描的手寫數字如下圖所示： Example in python 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 2...

deep learning Andrew學習筆記

Freezing one weight cause the same result with unfreezing（Deep learning書的一個練習）

智能推薦

猜你喜歡