Dtjytyk

Is this Paypal Github SDK reference really a dangerous site?

Sort array by month and year

What is Tony Stark injecting into himself in Iron Man 3?

What is the purpose of a disclaimer like "this is not legal advice"?

Will the concrete slab in a partially heated shed conduct a lot of heat to the unconditioned area?

School performs periodic password audits. Is my password compromised?

Too soon for a plot twist?

Issue with units for a rocket nozzle throat area problem

Can I challenge the interviewer to give me a proper technical feedback?

Is it appropriate to ask a former professor to order a library book for me through ILL?

Insult for someone who "doesn't know anything"

How spaceships determine each other's mass in space?

Should I file my taxes? No income, unemployed, but paid 2k in student loan interest

How do you use environments that have the same name within a single latex document?

Is divide-by-zero a security vulnerability?

Tabular environment - text vertically positions itself by bottom of tikz picture in adjacent cell

Is the differential, dp, exact or not?

If nine coins are tossed, what is the probability that the number of heads is even?

Why do phishing e-mails use faked e-mail addresses instead of the real one?

How to write a chaotic neutral protagonist and prevent my readers from thinking they are evil?

How to educate team mate to take screenshots for bugs with out unwanted stuff

Why would /etc/passwd be used every time someone executes `ls -l` command?

Should I apply for my boss's promotion?

Are small insurances worth it?

Proving the learnability of XOR function by a particular neural network

how to find the input for this optimization problem?Implement multiple input XOR with d-d-1 feed-forward neural networkWhat does the hypothesis function for a simple neural network (MLP with logistic activation, single output) look like in simplified form?Derivative of slope of neural network layer activationsApproximation rates of Neural NetworksWhat do these symbols mean regarding the stable points of a Hopfield network?Why is the Error surface for a 2 input neural network with 2 weights a parabolic bowlStudy the convexity of Mean Squared Error with regularizationIs my back propagation math correct?Are neural networks with bounded parameters a compact subset of the Banach space of continuous functions?

0

$begingroup$

Let's say I have the following neural network and the constraints:

1. The architecture is fixed (see the network in this image, I'm not allowed to post images due to low rep) (note that there are no biases)


2. Activation function for the hidden layer is $ReLU$ ;$ReLU(x) = max(0, x)$
   


3. There's no activation function for the output layer (should just return the sum of the inputs it receive).


4. Weights are constrained to be in the set ${-1, 0, 1}$
   

My question is:

Can we show if the XOR function is learnable or not given the network architecture and the associated constraints?

Here's how I thought about it:

Given the XOR truth table, we can right down equations for network output for each instance. If the inputs are $X_1$ and $X_2$ the output of the network $F(X_1, X_2)$ can be written as below in its general form:

$$ReLU(X_1w_1 + X_2w_3)w_5 + ReLU(X_1w_4 + X_2w_2)w_6 = F(X_1, X_2)$$

Using the truth table combinations, we obtain:

$0,1 rightarrow 1:$
$$max(0, 0 + 1.w_3)w_5 + max(0, 0 + 1w_2)w_6 = F(0, 1) = 1$$
$$max(0, w_3)w_5 + max(0, w_2)w_6 = 1 - (1)$$

$1,0 rightarrow 1:$
$$max(0, 1.w_1 + 0)w_5 + max(0, 1w_4 + 0)w_6 = F(1, 0) = 1$$
$$max(0, w_1)w_5 + max(0, w_4)w_6 = 1 - (2)$$

$1,1 rightarrow 0:$
$$max(0, 1.w_1 + 1.w_3)w_5 + max(0, 1w_4 + 1.w_2)w_6 = F(1, 1) = 0$$
$$max(0, w_1+w_3)w_5 + max(0, w_4+w_2)w_6 = 0 - (3)$$

Can we show that the above system of equations do/do not have a solution for $w_i$ values?

Here is a similar problem on crossvalidated.

proof-writing systems-of-equations constraints neural-networks constraint-programming

share|cite|improve this question

asked yesterday

j.Doej.Doe

1

New contributor

j.Doe is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.

$endgroup$

add a comment | 

0

$begingroup$

Let's say I have the following neural network and the constraints:

1. The architecture is fixed (see the network in this image, I'm not allowed to post images due to low rep) (note that there are no biases)


2. Activation function for the hidden layer is $ReLU$ ;$ReLU(x) = max(0, x)$
   


3. There's no activation function for the output layer (should just return the sum of the inputs it receive).


4. Weights are constrained to be in the set ${-1, 0, 1}$
   

My question is:

Can we show if the XOR function is learnable or not given the network architecture and the associated constraints?

Here's how I thought about it:

Given the XOR truth table, we can right down equations for network output for each instance. If the inputs are $X_1$ and $X_2$ the output of the network $F(X_1, X_2)$ can be written as below in its general form:

$$ReLU(X_1w_1 + X_2w_3)w_5 + ReLU(X_1w_4 + X_2w_2)w_6 = F(X_1, X_2)$$

Using the truth table combinations, we obtain:

$0,1 rightarrow 1:$
$$max(0, 0 + 1.w_3)w_5 + max(0, 0 + 1w_2)w_6 = F(0, 1) = 1$$
$$max(0, w_3)w_5 + max(0, w_2)w_6 = 1 - (1)$$

$1,0 rightarrow 1:$
$$max(0, 1.w_1 + 0)w_5 + max(0, 1w_4 + 0)w_6 = F(1, 0) = 1$$
$$max(0, w_1)w_5 + max(0, w_4)w_6 = 1 - (2)$$

$1,1 rightarrow 0:$
$$max(0, 1.w_1 + 1.w_3)w_5 + max(0, 1w_4 + 1.w_2)w_6 = F(1, 1) = 0$$
$$max(0, w_1+w_3)w_5 + max(0, w_4+w_2)w_6 = 0 - (3)$$

Can we show that the above system of equations do/do not have a solution for $w_i$ values?

Here is a similar problem on crossvalidated.

proof-writing systems-of-equations constraints neural-networks constraint-programming

share|cite|improve this question

asked yesterday

j.Doej.Doe

1

New contributor

j.Doe is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.

$endgroup$

add a comment | 

$begingroup$

Let's say I have the following neural network and the constraints:

1. The architecture is fixed (see the network in this image, I'm not allowed to post images due to low rep) (note that there are no biases)


2. Activation function for the hidden layer is $ReLU$ ;$ReLU(x) = max(0, x)$
   


3. There's no activation function for the output layer (should just return the sum of the inputs it receive).


4. Weights are constrained to be in the set ${-1, 0, 1}$
   

My question is:

Can we show if the XOR function is learnable or not given the network architecture and the associated constraints?

Here's how I thought about it:

Given the XOR truth table, we can right down equations for network output for each instance. If the inputs are $X_1$ and $X_2$ the output of the network $F(X_1, X_2)$ can be written as below in its general form:

$$ReLU(X_1w_1 + X_2w_3)w_5 + ReLU(X_1w_4 + X_2w_2)w_6 = F(X_1, X_2)$$

Using the truth table combinations, we obtain:

$0,1 rightarrow 1:$
$$max(0, 0 + 1.w_3)w_5 + max(0, 0 + 1w_2)w_6 = F(0, 1) = 1$$
$$max(0, w_3)w_5 + max(0, w_2)w_6 = 1 - (1)$$

$1,0 rightarrow 1:$
$$max(0, 1.w_1 + 0)w_5 + max(0, 1w_4 + 0)w_6 = F(1, 0) = 1$$
$$max(0, w_1)w_5 + max(0, w_4)w_6 = 1 - (2)$$

$1,1 rightarrow 0:$
$$max(0, 1.w_1 + 1.w_3)w_5 + max(0, 1w_4 + 1.w_2)w_6 = F(1, 1) = 0$$
$$max(0, w_1+w_3)w_5 + max(0, w_4+w_2)w_6 = 0 - (3)$$

Can we show that the above system of equations do/do not have a solution for $w_i$ values?

Here is a similar problem on crossvalidated.

proof-writing systems-of-equations constraints neural-networks constraint-programming

share|cite|improve this question

asked yesterday

j.Doej.Doe

1

New contributor

j.Doe is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.

$endgroup$

share|cite|improve this question

asked yesterday

j.Doej.Doe

1

New contributor

j.Doe is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.

share|cite|improve this question

asked yesterday

j.Doej.Doe

1

New contributor

j.Doe is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.

asked yesterday

j.Doej.Doe

1

New contributor

j.Doe is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.

asked yesterday

j.Doej.Doe

1