Skip to content

Commit 436ec9f

Browse files
mhjensenmhjensen
committed
update p1
1 parent 81c93cb commit 436ec9f

File tree

9 files changed

+2766
-33
lines changed

9 files changed

+2766
-33
lines changed

‎doc/Projects/2025/ODE/html/._ODE-bs000.html‎

Lines changed: 420 additions & 0 deletions
Large diffs are not rendered by default.

‎doc/Projects/2025/ODE/html/ODE-bs.html‎

Lines changed: 420 additions & 0 deletions
Large diffs are not rendered by default.

‎doc/Projects/2025/ODE/html/ODE.html‎

Lines changed: 452 additions & 0 deletions
Large diffs are not rendered by default.

‎doc/Projects/2025/ODE/ipynb/ODE.ipynb‎

Lines changed: 390 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,390 @@
1+
{
2+
"cells": [
3+
{
4+
"cell_type": "markdown",
5+
"id": "818acc1d",
6+
"metadata": {
7+
"editable": true
8+
},
9+
"source": [
10+
"<!-- HTML file automatically generated from DocOnce source (https://github.com/doconce/doconce/)\n",
11+
"doconce format html ODE.do.txt -->\n",
12+
"<!-- dom:TITLE: Project 1, Ordinary differential equations with Neural Networks and Recurrent Neural Networks -->"
13+
]
14+
},
15+
{
16+
"cell_type": "markdown",
17+
"id": "0ec82088",
18+
"metadata": {
19+
"editable": true
20+
},
21+
"source": [
22+
"# Project 1, Ordinary differential equations with Neural Networks and Recurrent Neural Networks\n",
23+
"**[FYS5429/9429](https://www.uio.no/studier/emner/matnat/fys/FYS5429/index-eng.html), Advanced machine learning and data analysis for the physical sciences, University of Oslo, Norway**\n",
24+
"\n",
25+
"Date: **Spring semester 2025, deadline March 21**"
26+
]
27+
},
28+
{
29+
"cell_type": "markdown",
30+
"id": "816d79db",
31+
"metadata": {
32+
"editable": true
33+
},
34+
"source": [
35+
"## Solving differential equations with neural networks\n",
36+
"\n",
37+
"Here we describe the possible differential equations we can study\n",
38+
"first with neural networks and thereafter with recurrent neural\n",
39+
"networks and/or Autoenconders. Feel free to use own codes or TensorFlow/Keras.PyTorch or toher libraries.\n",
40+
"Furthermore, you can change the type of differential equations.\n",
41+
"\n",
42+
"The differential equations are given by the so-called [Lorenz attractor model](https://encyclopediaofmath.org/index.php?title=Lorenz_attractor), and read"
43+
]
44+
},
45+
{
46+
"cell_type": "markdown",
47+
"id": "9060bc0c",
48+
"metadata": {
49+
"editable": true
50+
},
51+
"source": [
52+
"$$\n",
53+
"\\frac{dx}{dt}=\\sigma\\left(y-x\\right),\n",
54+
"$$"
55+
]
56+
},
57+
{
58+
"cell_type": "markdown",
59+
"id": "fce8d016",
60+
"metadata": {
61+
"editable": true
62+
},
63+
"source": [
64+
"where $\\sigma =10$ is a constant"
65+
]
66+
},
67+
{
68+
"cell_type": "markdown",
69+
"id": "df47845b",
70+
"metadata": {
71+
"editable": true
72+
},
73+
"source": [
74+
"$$\n",
75+
"\\frac{dy}{dt}= x\\left(\\rho-z\\right)-y,\n",
76+
"$$"
77+
]
78+
},
79+
{
80+
"cell_type": "markdown",
81+
"id": "1f110b1d",
82+
"metadata": {
83+
"editable": true
84+
},
85+
"source": [
86+
"with $\\rho=28$ and"
87+
]
88+
},
89+
{
90+
"cell_type": "markdown",
91+
"id": "cd04015a",
92+
"metadata": {
93+
"editable": true
94+
},
95+
"source": [
96+
"$$\n",
97+
"\\frac{dz}{dt}=xy-\\beta z\n",
98+
"$$"
99+
]
100+
},
101+
{
102+
"cell_type": "markdown",
103+
"id": "5d293665",
104+
"metadata": {
105+
"editable": true
106+
},
107+
"source": [
108+
"with $\\beta=8/3$ as our final constant.\n",
109+
"\n",
110+
"The following function is a\n",
111+
"simple function which sets up the solution using the ordinary\n",
112+
"differential library which follows **NumPy**. Here we have fixed the\n",
113+
"time sted $\\Delta t=0.01$ and the final time $t_f=8$.\n",
114+
"\n",
115+
"The program sets $100$ random initial values and produces inputs and outputs for a neural network calculations.\n",
116+
"The inputs are given by the values of the array $\\boldsymbol{x}$ (which contains $x,y,z$ as functions of time) for the time step $\\boldsymbol{x}_t$.\n",
117+
"The other array defined by $\\boldsymbol{x}_{t+1}$ contains the outputs (or targets) which we want the neural network to reproduce."
118+
]
119+
},
120+
{
121+
"cell_type": "code",
122+
"execution_count": 1,
123+
"id": "a53e0bb8",
124+
"metadata": {
125+
"collapsed": false,
126+
"editable": true
127+
},
128+
"outputs": [],
129+
"source": [
130+
"%matplotlib inline\n",
131+
"\n",
132+
"# Common imports\n",
133+
"import numpy as np\n",
134+
"from scipy.integrate import odeint\n",
135+
"import matplotlib.pyplot as plt\n",
136+
"import os\n",
137+
"\n",
138+
"# Where to save the figures and data files\n",
139+
"PROJECT_ROOT_DIR = \"Results\"\n",
140+
"FIGURE_ID = \"Results/FigureFiles\"\n",
141+
"DATA_ID = \"DataFiles/\"\n",
142+
"\n",
143+
"if not os.path.exists(PROJECT_ROOT_DIR):\n",
144+
" os.mkdir(PROJECT_ROOT_DIR)\n",
145+
"\n",
146+
"if not os.path.exists(FIGURE_ID):\n",
147+
" os.makedirs(FIGURE_ID)\n",
148+
"\n",
149+
"if not os.path.exists(DATA_ID):\n",
150+
" os.makedirs(DATA_ID)\n",
151+
"\n",
152+
"def image_path(fig_id):\n",
153+
" return os.path.join(FIGURE_ID, fig_id)\n",
154+
"\n",
155+
"def data_path(dat_id):\n",
156+
" return os.path.join(DATA_ID, dat_id)\n",
157+
"\n",
158+
"def save_fig(fig_id):\n",
159+
" plt.savefig(image_path(fig_id) + \".png\", format='png')\n",
160+
"\n",
161+
"\n",
162+
"# Selection of parameter values and setting array for time\n",
163+
"dt =0.01; tfinal = 8\n",
164+
"t = np.arange(0,tfinal+dt, dt)\n",
165+
"beta =8.0/3.0; rho = 28.0; sigma = 10.0\n",
166+
"\n",
167+
"# define the inputs and outputs for the neural networks\n",
168+
"nninput = np.zeros((100*len(t)-1,3))\n",
169+
"nnoutput = np.zeros((100*len(t)-1,3))\n",
170+
"# Define the equations to integrate\n",
171+
"def lorenz_derivative(xyz, t0, sigma=sigma,beta=beta,rho=rho):\n",
172+
" x, y, z = xyz\n",
173+
" return [sigma*(x-y), x*(rho-z)-y, x*y-beta*z]\n",
174+
"\n",
175+
"# generate 100 random initial values\n",
176+
"x0 = -15.0+30.0*np.random.random((100,3))\n",
177+
"\n",
178+
"# Use odeint functionality by sending in derivative function\n",
179+
"# Feel free to change the choice of integrator\n",
180+
"x_t = np.asarray([odeint(lorenz_derivative, x0_j, t) \n",
181+
" for x0_j in x0])\n",
182+
"\n",
183+
"# define the inputs and outputs for the neural networks\n",
184+
"for j in range(100):\n",
185+
" nninput[j*(len(t)-1):(j+1)*(len(t)-1),:] = x_t[j,:-1,:]\n",
186+
" nnoutput[j*(len(t)-1):(j+1)*(len(t)-1),:] = x_t[j,1:,:]"
187+
]
188+
},
189+
{
190+
"cell_type": "markdown",
191+
"id": "7b1fe53f",
192+
"metadata": {
193+
"editable": true
194+
},
195+
"source": [
196+
"The input and output variables are those we will start trying our\n",
197+
"network with. Your first taks is to set up a neural code (either using\n",
198+
"your own code or TensorFlow/PyTorch or similar libraries)) and use the\n",
199+
"above data to a prediction for the time evolution of Lorenz system for\n",
200+
"various values of the randomly chosen initial values. Study the\n",
201+
"dependence of the fit as function of the architecture of the network\n",
202+
"(number of nodes, hidden layers and types of activation functions) and\n",
203+
"various regularization schemes and optimization methods like standard\n",
204+
"gradient descent with momentum, stochastic gradient descent with\n",
205+
"batches and with and without momentum and various schedulers for the\n",
206+
"learning rate.\n",
207+
"\n",
208+
"Feel free to change the above differential equations. As an example,\n",
209+
"consider the following harmonic oscillator equations solved with the\n",
210+
"Runge-Kutta to fourth order method. This is a one-dimensional problem\n",
211+
"and it produces a position $x_t$ and velocity $v_t$. You could now try\n",
212+
"to fit both the velocities and positions using much of the same recipe\n",
213+
"as for Lorenz attractor. You will find it convenient to analyze one\n",
214+
"set of initial conditions first. The code is included here.\n",
215+
"\n",
216+
"This code is an example code that solves Newton's equations of motion\n",
217+
"with a given force and produces an output which in turn can be used to\n",
218+
"train a neural network"
219+
]
220+
},
221+
{
222+
"cell_type": "code",
223+
"execution_count": 2,
224+
"id": "80d3ff0a",
225+
"metadata": {
226+
"collapsed": false,
227+
"editable": true
228+
},
229+
"outputs": [],
230+
"source": [
231+
"# Common imports\n",
232+
"import numpy as np\n",
233+
"import pandas as pd\n",
234+
"from math import *\n",
235+
"import matplotlib.pyplot as plt\n",
236+
"import os\n",
237+
"\n",
238+
"# Where to save the figures and data files\n",
239+
"PROJECT_ROOT_DIR = \"Results\"\n",
240+
"FIGURE_ID = \"Results/FigureFiles\"\n",
241+
"DATA_ID = \"DataFiles/\"\n",
242+
"\n",
243+
"if not os.path.exists(PROJECT_ROOT_DIR):\n",
244+
" os.mkdir(PROJECT_ROOT_DIR)\n",
245+
"\n",
246+
"if not os.path.exists(FIGURE_ID):\n",
247+
" os.makedirs(FIGURE_ID)\n",
248+
"\n",
249+
"if not os.path.exists(DATA_ID):\n",
250+
" os.makedirs(DATA_ID)\n",
251+
"\n",
252+
"def image_path(fig_id):\n",
253+
" return os.path.join(FIGURE_ID, fig_id)\n",
254+
"\n",
255+
"def data_path(dat_id):\n",
256+
" return os.path.join(DATA_ID, dat_id)\n",
257+
"\n",
258+
"def save_fig(fig_id):\n",
259+
" plt.savefig(image_path(fig_id) + \".png\", format='png')\n",
260+
"\n",
261+
"\n",
262+
"def SpringForce(v,x,t):\n",
263+
"# note here that we have divided by mass and we return the acceleration\n",
264+
" return -2*gamma*v-x+Ftilde*cos(t*Omegatilde)\n",
265+
"\n",
266+
"\n",
267+
"def RK4(v,x,t,n,Force):\n",
268+
" for i in range(n-1):\n",
269+
"# Setting up k1\n",
270+
" k1x = DeltaT*v[i]\n",
271+
" k1v = DeltaT*Force(v[i],x[i],t[i])\n",
272+
"# Setting up k2\n",
273+
" vv = v[i]+k1v*0.5\n",
274+
" xx = x[i]+k1x*0.5\n",
275+
" k2x = DeltaT*vv\n",
276+
" k2v = DeltaT*Force(vv,xx,t[i]+DeltaT*0.5)\n",
277+
"# Setting up k3\n",
278+
" vv = v[i]+k2v*0.5\n",
279+
" xx = x[i]+k2x*0.5\n",
280+
" k3x = DeltaT*vv\n",
281+
" k3v = DeltaT*Force(vv,xx,t[i]+DeltaT*0.5)\n",
282+
"# Setting up k4\n",
283+
" vv = v[i]+k3v\n",
284+
" xx = x[i]+k3x\n",
285+
" k4x = DeltaT*vv\n",
286+
" k4v = DeltaT*Force(vv,xx,t[i]+DeltaT)\n",
287+
"# Final result\n",
288+
" x[i+1] = x[i]+(k1x+2*k2x+2*k3x+k4x)/6.\n",
289+
" v[i+1] = v[i]+(k1v+2*k2v+2*k3v+k4v)/6.\n",
290+
" t[i+1] = t[i] + DeltaT\n",
291+
"\n",
292+
"\n",
293+
"# Main part begins here\n",
294+
"\n",
295+
"DeltaT = 0.001\n",
296+
"#set up arrays \n",
297+
"tfinal = 20 # in dimensionless time\n",
298+
"n = ceil(tfinal/DeltaT)\n",
299+
"# set up arrays for t, v, and x\n",
300+
"t = np.zeros(n)\n",
301+
"v = np.zeros(n)\n",
302+
"x = np.zeros(n)\n",
303+
"# Initial conditions (can change to more than one dim)\n",
304+
"x0 = 1.0 \n",
305+
"v0 = 0.0\n",
306+
"x[0] = x0\n",
307+
"v[0] = v0\n",
308+
"gamma = 0.2\n",
309+
"Omegatilde = 0.5\n",
310+
"Ftilde = 1.0\n",
311+
"# Start integrating using Euler's method\n",
312+
"# Note that we define the force function as a SpringForce\n",
313+
"RK4(v,x,t,n,SpringForce)\n",
314+
"\n",
315+
"# Plot position as function of time \n",
316+
"fig, ax = plt.subplots()\n",
317+
"ax.set_ylabel('x[m]')\n",
318+
"ax.set_xlabel('t[s]')\n",
319+
"ax.plot(t, x)\n",
320+
"fig.tight_layout()\n",
321+
"save_fig(\"ForcedBlockRK4\")\n",
322+
"plt.show()"
323+
]
324+
},
325+
{
326+
"cell_type": "markdown",
327+
"id": "a19b1bc3",
328+
"metadata": {
329+
"editable": true
330+
},
331+
"source": [
332+
"The next step is to include recurrent neural networks."
333+
]
334+
},
335+
{
336+
"cell_type": "markdown",
337+
"id": "553ddeaf",
338+
"metadata": {
339+
"editable": true
340+
},
341+
"source": [
342+
"## Introduction to numerical projects\n",
343+
"\n",
344+
"Here follows a brief recipe and recommendation on how to write a report for each\n",
345+
"project.\n",
346+
"\n",
347+
" * Give a short description of the nature of the problem and the eventual numerical methods you have used.\n",
348+
"\n",
349+
" * Describe the algorithm you have used and/or developed. Here you may find it convenient to use pseudocoding. In many cases you can describe the algorithm in the program itself.\n",
350+
"\n",
351+
" * Include the source code of your program. Comment your program properly.\n",
352+
"\n",
353+
" * If possible, try to find analytic solutions, or known limits in order to test your program when developing the code.\n",
354+
"\n",
355+
" * Include your results either in figure form or in a table. Remember to label your results. All tables and figures should have relevant captions and labels on the axes.\n",
356+
"\n",
357+
" * Try to evaluate the reliabilty and numerical stability/precision of your results. If possible, include a qualitative and/or quantitative discussion of the numerical stability, eventual loss of precision etc.\n",
358+
"\n",
359+
" * Try to give an interpretation of you results in your answers to the problems.\n",
360+
"\n",
361+
" * Critique: if possible include your comments and reflections about the exercise, whether you felt you learnt something, ideas for improvements and other thoughts you've made when solving the exercise. We wish to keep this course at the interactive level and your comments can help us improve it.\n",
362+
"\n",
363+
" * Try to establish a practice where you log your work at the computerlab. You may find such a logbook very handy at later stages in your work, especially when you don't properly remember what a previous test version of your program did. Here you could also record the time spent on solving the exercise, various algorithms you may have tested or other topics which you feel worthy of mentioning."
364+
]
365+
},
366+
{
367+
"cell_type": "markdown",
368+
"id": "54c2a55c",
369+
"metadata": {
370+
"editable": true
371+
},
372+
"source": [
373+
"## Format for electronic delivery of report and programs\n",
374+
"\n",
375+
"The preferred format for the report is a PDF file. You can also use DOC or postscript formats or as an ipython notebook file. As programming language we prefer that you choose between C/C++, Fortran2008 or Python. The following prescription should be followed when preparing the report:\n",
376+
"\n",
377+
" * Send us an email in order to hand in your projects with a link to your GitHub/Gitlab repository.\n",
378+
"\n",
379+
" * In your GitHub/GitLab or similar repository, please include a folder which contains selected results. These can be in the form of output from your code for a selected set of runs and input parameters.\n",
380+
"\n",
381+
"Finally, \n",
382+
"we encourage you to collaborate. Optimal working groups consist of \n",
383+
"2-3 students. You can then hand in a common report."
384+
]
385+
}
386+
],
387+
"metadata": {},
388+
"nbformat": 4,
389+
"nbformat_minor": 5
390+
}

‎doc/Projects/2025/ODE/ipynb/ipynb-ODE-src.tar.gz‎

191 Bytes
Binary file not shown.

0 commit comments

Comments
 (0)