In Gluon, the fully connected layer is defined in the Dense class. Since we only want to generate a single scalar output, we set that number to 1. It is worth noting that, for convenience, Gluon does not require us to specify the input shape for each layer. Hence we do not need to tell Gluon how many inputs go into this linear layer. When we first pass data through our model, e.g., when we execute net(X) later, Gluon will automatically infer the number of inputs to each layer and thus instantiate the correct model. We will describe how this works in more detail later.

class LinearRegression(d2l.Module): #@save
  """The linear regression model implemented with high-level APIs."""
  def __init__(self, lr):
    super().__init__()
    self.save_hyperparameters()
    self.net = nn.Dense(1)
    self.net.initialize(init.Normal(sigma=0.01))

In Keras, the fully connected layer is defined in the Dense class. Since we only want to generate a single scalar output, we set that number to 1. It is worth noting that, for convenience, Keras does not require us to specify the input shape for each layer. We do not need to tell Keras how many inputs go into this linear layer. When we first try to pass data through our model, e.g., when we execute net(X) later, Keras will automatically infer the number of inputs to each layer. We will describe how this works in more detail later.

class LinearRegression(d2l.Module): #@save
  """The linear regression model implemented with high-level APIs."""
  def __init__(self, lr):
    super().__init__()
    self.save_hyperparameters()
    initializer = tf.initializers.RandomNormal(stddev=0.01)
    self.net = tf.keras.layers.Dense(1, kernel_initializer=initializer)

The loss module defines many useful loss functions. For speed and convenience, we forgo implementing our own and choose the built-in loss.L2Loss instead. Because the loss that it returns is the squared error for each example, we use meanto average the loss across over the minibatch.

@d2l.add_to_class(LinearRegression) #@save
def loss(self, y_hat, y):
  fn = gluon.loss.L2Loss()
  return fn(y_hat, y).mean()

The MeanSquaredError class computes the mean squared error (without the 1/2 factor in (3.1.5)). By default, it returns the average loss over examples.

@d2l.add_to_class(LinearRegression) #@save
def loss(self, y_hat, y):
  fn = tf.keras.losses.MeanSquaredError()
  return fn(y, y_hat)