带张量流的 SVM

数据挖掘 分类 张量流 支持向量机
2022-03-07 22:49:24

我有一个包含以下数据的 Numpy 数组,例如:

['13 .398249765480822 ''19 .324784598731966' '80 .98629514090669 '
  '-3.703122956721927e-06' '80 .98629884402965 ''24 .008452881790028'
  '679.6408224307851' '2498.8247399799975', 'fear']

另一个具有相同长度和不同数字的 Numpy 数组以及另一个“中性”标签。

事实是我正在使用 Github 的代码(Setosa)和其他文章来制作二进制分类器(恐惧或中立),但我收到以下错误,因为我不知道该怎么做,所以我考虑了所有数组中的数字而不是 Setosa 的代码,它在执行网格时只考虑两个。

## SVM con Tensorflow
sess = tf.Session()
x_vals = np.array([[x[0], x[1], x[2], x[3], x[4], x[5], x[6], x[7]] for x in matrix])
y_vals = np.array([1 if y[8] == 'fear' else -1 for y in matrix])

# Split the train data and testing data
train_indices = np.random.choice(len(x_vals), int(round(len(x_vals)*0.8)), replace=False)
test_indices = np.array(list(set(range(len(x_vals))) - set(train_indices)))
x_vals_train = x_vals[train_indices]
x_vals_test = x_vals[test_indices]
y_vals_train = y_vals[train_indices]
y_vals_test = y_vals[test_indices]

class1_x = [x[0] for i, x in enumerate(x_vals_train) if y_vals_train[i] == 1]
class1_y = [x[1] for i, x in enumerate(x_vals_train) if y_vals_train[i] == 1]
class2_x = [x[0] for i, x in enumerate(x_vals_train) if y_vals_train[i] == -1]
class2_y = [x[1] for i, x in enumerate(x_vals_train) if y_vals_train[i] == -1]

# Declare batch size
batch_size = 150

# Initialize placeholders
x_data = tf.placeholder(shape=[None, 8], dtype=tf.float32)
y_target = tf.placeholder(shape=[None, 1], dtype=tf.float32)
prediction_grid = tf.placeholder(shape=[None, 8], dtype=tf.float32)

# Create variables for svm
b = tf.Variable(tf.random_normal(shape=[1, batch_size]))

# Gaussian (RBF) kernel
gamma = tf.constant(-10.0)
sq_dists = tf.multiply(2., tf.matmul(x_data, tf.transpose(x_data)))
my_kernel = tf.exp(tf.multiply(gamma, tf.abs(sq_dists)))

# Compute SVM Model
first_term = tf.reduce_sum(b)
b_vec_cross = tf.matmul(tf.transpose(b), b)
y_target_cross = tf.matmul(y_target, tf.transpose(y_target))
second_term = tf.reduce_sum(tf.multiply(my_kernel, tf.multiply(b_vec_cross, y_target_cross)))
loss = tf.negative(tf.subtract(first_term, second_term))

# Gaussian (RBF) prediction kernel
rA = tf.reshape(tf.reduce_sum(tf.square(x_data), 1), [-1, 1])
rB = tf.reshape(tf.reduce_sum(tf.square(prediction_grid), 1), [-1, 1])
pred_sq_dist = tf.add(tf.subtract(rA, tf.multiply(2., tf.matmul(x_data, tf.transpose(prediction_grid)))), tf.transpose(rB))
pred_kernel = tf.exp(tf.multiply(gamma, tf.abs(pred_sq_dist)))

prediction_output = tf.matmul(tf.multiply(tf.transpose(y_target), b), pred_kernel)
prediction = tf.sign(prediction_output - tf.reduce_mean(prediction_output))
accuracy = tf.reduce_mean(tf.cast(tf.equal(tf.squeeze(prediction), tf.squeeze(y_target)), tf.float32))

# Declare optimizer
my_opt = tf.train.GradientDescentOptimizer(0.01)
train_step = my_opt.minimize(loss)

# Initialize variables
init = tf.global_variables_initializer()
sess.run(init)

# Training loop
loss_vec = []
batch_accuracy = []
for i in range(300):
    rand_index = np.random.choice(len(x_vals), size=batch_size)
    rand_x = x_vals[rand_index]
    rand_y = np.transpose([y_vals[rand_index]])
    sess.run(train_step, feed_dict={x_data: rand_x, y_target: rand_y})

    temp_loss = sess.run(loss, feed_dict={x_data: rand_x, y_target: rand_y})
    loss_vec.append(temp_loss)

    acc_temp = sess.run(accuracy, feed_dict={x_data: rand_x,
                                             y_target: rand_y,
                                             prediction_grid: rand_x})
    batch_accuracy.append(acc_temp)

    if (i + 1) % 75 == 0:
        print('Step #' + str(i + 1))
        print('Loss = ' + str(temp_loss))

# Create a mesh to plot points in
x_vals = x_vals.astype(np.float)
x_min, x_max = x_vals[:, 0].min() - 1, x_vals[:, 0].max() + 1
y_min, y_max = x_vals[:, 1].min() - 1, x_vals[:, 1].max() + 1
xx, yy = np.meshgrid(np.arange(x_min, x_max, 0.02),
                     np.arange(y_min, y_max, 0.02))
grid_points = np.c_[xx.ravel(), yy.ravel()]
[grid_predictions] = sess.run(prediction, feed_dict={x_data: x_vals,
                                                     y_target: np.transpose([y_vals]),
                                                     prediction_grid: grid_points})
grid_predictions = grid_predictions.reshape(xx.shape)

# Plot points and grid
plt.contourf(xx, yy, grid_predictions, cmap=plt.cm.Paired, alpha=0.8)
plt.plot(class1_x, class1_y, 'ro', label='I. setosa')
plt.plot(class2_x, class2_y, 'kx', label='Non setosa')
plt.title('Gaussian SVM Results on Iris Data')
plt.xlabel('Petal Length')
plt.ylabel('Sepal Width')
plt.legend(loc='lower right')
plt.ylim([-0.5, 3.0])
plt.xlim([3.5, 8.5])
plt.show()

# Plot batch accuracy
plt.plot(batch_accuracy, 'k-', label='Accuracy')
plt.title('Batch Accuracy')
plt.xlabel('Generation')
plt.ylabel('Accuracy')
plt.legend(loc='lower right')
plt.show()

# Plot loss over time
plt.plot(loss_vec, 'k-')
plt.title('Loss per Generation')
plt.xlabel('Generation')
plt.ylabel('Loss')
plt.show()

得到的错误是:

File "test.py", line 154, in <module>
    prediction_grid: grid_points})
ValueError: Cannot feed value of shape (30119320, 2) for Tensor u'Placeholder_2:0', which has shape '(?, 8)'

我知道它们的形状不同,但我不知道如何改变它或做什么,因为我需要制作一个具有 8 个特征和两个类“中性”和“恐惧”的分类器。

原始代码在这里

1个回答

此代码仅用于 2D 输入,不能用于 8D 输入。

这是tensorflow 的 SVM 的 stackoverflow示例tf.contrib.learn.SVM

此外,这是一个易于使用的 Python 中的SVM 示例(没有 tensorflow)。

关于代码

2D 假设被深度集成到prediction_grid变量和绘图的代码中。

一个重要的部分是何时需要创建网格:

x_min, x_max = x_vals[:, 0].min() - 1, x_vals[:, 0].max() + 1
y_min, y_max = x_vals[:, 1].min() - 1, x_vals[:, 1].max() + 1
xx, yy = np.meshgrid(np.arange(x_min, x_max, 0.02),
                     np.arange(y_min, y_max, 0.02))
grid_points = np.c_[xx.ravel(), yy.ravel()]

这创建了一个1502×2 grid_points. 该网格稍后用于 2D 绘图。由于grid_points尺寸是150d×d,它为 8D 加注MemoryError(甚至为 4D)。

这是我用来试验更高维度的代码的修改版本。Memory Error通过将网格步长从 0.02 更改为 1 来避免,从而减少150d3d(增加grid_step更大范围的输入)。

import tensorflow as tf
import numpy as np
import matplotlib.pyplot as plt

dimension = 8
N = 300
grid_step = 1  # default value was 0.02

x_dummy = np.random.random((N, dimension))
y_dummy = np.random.choice(['fear', 'abc'], (N, 1))
matrix = np.hstack((x_dummy, y_dummy))

## SVM con Tensorflow
sess = tf.Session()
x_vals = np.array([x[0:dimension] for x in matrix])
y_vals = np.array([1 if y[dimension] == 'fear' else -1 for y in matrix])

# Split the train data and testing data
train_indices = np.random.choice(len(x_vals), int(round(len(x_vals)*0.8)), replace=False)
test_indices = np.array(list(set(range(len(x_vals))) - set(train_indices)))
x_vals_train = x_vals[train_indices]
x_vals_test = x_vals[test_indices]
y_vals_train = y_vals[train_indices]
y_vals_test = y_vals[test_indices]

class1_x = [x[0] for i, x in enumerate(x_vals_train) if y_vals_train[i] == 1]
class1_y = [x[1] for i, x in enumerate(x_vals_train) if y_vals_train[i] == 1]
class2_x = [x[0] for i, x in enumerate(x_vals_train) if y_vals_train[i] == -1]
class2_y = [x[1] for i, x in enumerate(x_vals_train) if y_vals_train[i] == -1]

# Declare batch size
batch_size = N

# Initialize placeholders
x_data = tf.placeholder(shape=[None, dimension], dtype=tf.float32)
y_target = tf.placeholder(shape=[None, 1], dtype=tf.float32)
prediction_grid = tf.placeholder(shape=[None, dimension], dtype=tf.float32)

# Create variables for svm
b = tf.Variable(tf.random_normal(shape=[1, batch_size]))

# Gaussian (RBF) kernel
gamma = tf.constant(-10.0)
sq_dists = tf.multiply(2., tf.matmul(x_data, tf.transpose(x_data)))
my_kernel = tf.exp(tf.multiply(gamma, tf.abs(sq_dists)))

# Compute SVM Model
first_term = tf.reduce_sum(b)
b_vec_cross = tf.matmul(tf.transpose(b), b)
y_target_cross = tf.matmul(y_target, tf.transpose(y_target))
second_term = tf.reduce_sum(tf.multiply(my_kernel, tf.multiply(b_vec_cross, y_target_cross)))
loss = tf.negative(tf.subtract(first_term, second_term))

# Gaussian (RBF) prediction kernel
rA = tf.reshape(tf.reduce_sum(tf.square(x_data), 1), [-1, 1])
rB = tf.reshape(tf.reduce_sum(tf.square(prediction_grid), 1), [-1, 1])
pred_sq_dist = tf.add(tf.subtract(rA, tf.multiply(2., tf.matmul(x_data, tf.transpose(prediction_grid)))), tf.transpose(rB))
pred_kernel = tf.exp(tf.multiply(gamma, tf.abs(pred_sq_dist)))

prediction_output = tf.matmul(tf.multiply(tf.transpose(y_target), b), pred_kernel)
prediction = tf.sign(prediction_output - tf.reduce_mean(prediction_output))
accuracy = tf.reduce_mean(tf.cast(tf.equal(tf.squeeze(prediction), tf.squeeze(y_target)), tf.float32))

# Declare optimizer
my_opt = tf.train.GradientDescentOptimizer(0.01)
train_step = my_opt.minimize(loss)

# Initialize variables
init = tf.global_variables_initializer()
sess.run(init)

# Training loop
loss_vec = []
batch_accuracy = []
for i in range(300):
    rand_index = np.random.choice(len(x_vals), size=batch_size)
    rand_x = x_vals[rand_index]
    rand_y = np.transpose([y_vals[rand_index]])
    sess.run(train_step, feed_dict={x_data: rand_x, y_target: rand_y})

    temp_loss = sess.run(loss, feed_dict={x_data: rand_x, y_target: rand_y})
    loss_vec.append(temp_loss)

    acc_temp = sess.run(accuracy, feed_dict={x_data: rand_x,
                                             y_target: rand_y,
                                             prediction_grid: rand_x})
    batch_accuracy.append(acc_temp)

    if (i + 1) % 75 == 0:
        print('Step #' + str(i + 1))
        print('Loss = ' + str(temp_loss))

# Create a mesh to plot points in
x_vals = x_vals.astype(np.float)
# this code is used as a generalization to work with all dimensions
x_ranges = np.vstack((x_vals.min(axis=0) - 1, x_vals.max(axis=0) + 1)).T
aranges = [np.arange(x_range[0], x_range[1], grid_step) for x_range in x_ranges]
print('grid size:', np.power(len(aranges[0]), dimension))
meshes = np.meshgrid(*aranges)
grid_points = np.vstack(tuple([mesh.ravel() for mesh in meshes])).T
print('grid size:', grid_points.shape)
[grid_predictions] = sess.run(prediction, feed_dict={x_data: x_vals,
                                                     y_target: np.transpose([y_vals]),
                                                     prediction_grid: grid_points})

# Plot points and grid
# this is the old mesh generation code that is kept since it is used in the plots
x_min, x_max = x_vals[:, 0].min() - 1, x_vals[:, 0].max() + 1
y_min, y_max = x_vals[:, 1].min() - 1, x_vals[:, 1].max() + 1
xx_arange = np.arange(x_min, x_max, grid_step)
yy_arange = np.arange(y_min, y_max, grid_step)
xx, yy = np.meshgrid(xx_arange,yy_arange)
size = np.power(len(xx), 2)
grid_predictions = grid_predictions[0:size].reshape(xx.shape)

plt.contourf(xx, yy, grid_predictions, cmap=plt.cm.Paired, alpha=0.8)
plt.plot(class1_x, class1_y, 'ro', label='I. setosa')
plt.plot(class2_x, class2_y, 'kx', label='Non setosa')
plt.title('Gaussian SVM Results on Iris Data')
plt.xlabel('Petal Length')
plt.ylabel('Sepal Width')
plt.legend(loc='lower right')
plt.ylim([-0.5, 3.0])
plt.xlim([3.5, 8.5])
plt.show()

# Plot batch accuracy
plt.plot(batch_accuracy, 'k-', label='Accuracy')
plt.title('Batch Accuracy')
plt.xlabel('Generation')
plt.ylabel('Accuracy')
plt.legend(loc='lower right')
plt.show()

# Plot loss over time
plt.plot(loss_vec, 'k-')
plt.title('Loss per Generation')
plt.xlabel('Generation')
plt.ylabel('Loss')
plt.show()

输出:

Step #75
Loss = -251.9497
Step #150
Loss = -476.96854
Step #225
Loss = -701.92444
Step #300
Loss = -927.2843
grid size: 6561
grid size: (6561, 8)