Download Least-Square Data Fitting in Advanced Linear Algebra and more Lecture notes Machine Learning in PDF only on Docsity!
𝒏
$
, Hilbert space (vector space of infinite dimension)
a
T
b =
∞
i
a
i
b
i
dxa ( x ) b ( x )
0 10 20 30 40 50
0
π
4
π
2
3 π
4
π
0
π
4
π
2
3 π
4 π
cos x
AAAB/nicbVDLSgMxFL1TX3V8VV26CZaCqzIjgi6LblxWsA9oS8lkMm1skhmSjFiGgp/gVj/Anbj1V1z7I6bTLmzrgQuHc+5N7j1Bwpk2nvftFNbWNza3itvuzu7e/kHp8Kip41QR2iAxj1U7wJpyJmnDMMNpO1EUi4DTVjC6mfqtR6o0i+W9GSe0J/BAsogRbKzU7JJYo6d+qexVvRxolfhzUq4VIUe9X/rphjFJBZWGcKx1x/cS08uwMoxwOnG7qaYJJiM8oB1LJRZU97J82wmqWCVEUaxsSYNy9e9EhoXWYxHYToHNUC97U/E/r5Oa6KqXMZmkhkoy+yhKOTIxmp6OQqYoMXy88GAgJm4lZJgoZpdHZIgVJsYm5tpU/OUMVknzvOpbfndRrl3P4oEinMApnIEPl1CDW6hDAwg8wAu8wpvz7Lw7H87nrLXgzGeOYQHO1y+IP5XyAAAB/nicbVDLSgMxFL1TX3V8VV26CZaCqzIjgi6LblxWsA9oS8lkMm1skhmSjFiGgp/gVj/Anbj1V1z7I6bTLmzrgQuHc+5N7j1Bwpk2nvftFNbWNza3itvuzu7e/kHp8Kip41QR2iAxj1U7wJpyJmnDMMNpO1EUi4DTVjC6mfqtR6o0i+W9GSe0J/BAsogRbKzU7JJYo6d+qexVvRxolfhzUq4VIUe9X/rphjFJBZWGcKx1x/cS08uwMoxwOnG7qaYJJiM8oB1LJRZU97J82wmqWCVEUaxsSYNy9e9EhoXWYxHYToHNUC97U/E/r5Oa6KqXMZmkhkoy+yhKOTIxmp6OQqYoMXy88GAgJm4lZJgoZpdHZIgVJsYm5tpU/OUMVknzvOpbfndRrl3P4oEinMApnIEPl1CDW6hDAwg8wAu8wpvz7Lw7H87nrLXgzGeOYQHO1y+IP5XyAAAB/nicbVDLSgMxFL1TX3V8VV26CZaCqzIjgi6LblxWsA9oS8lkMm1skhmSjFiGgp/gVj/Anbj1V1z7I6bTLmzrgQuHc+5N7j1Bwpk2nvftFNbWNza3itvuzu7e/kHp8Kip41QR2iAxj1U7wJpyJmnDMMNpO1EUi4DTVjC6mfqtR6o0i+W9GSe0J/BAsogRbKzU7JJYo6d+qexVvRxolfhzUq4VIUe9X/rphjFJBZWGcKx1x/cS08uwMoxwOnG7qaYJJiM8oB1LJRZU97J82wmqWCVEUaxsSYNy9e9EhoXWYxHYToHNUC97U/E/r5Oa6KqXMZmkhkoy+yhKOTIxmp6OQqYoMXy88GAgJm4lZJgoZpdHZIgVJsYm5tpU/OUMVknzvOpbfndRrl3P4oEinMApnIEPl1CDW6hDAwg8wAu8wpvz7Lw7H87nrLXgzGeOYQHO1y+IP5XyAAAB/nicbVDLSgMxFL1TX3V8VV26CZaCqzIjgi6LblxWsA9oS8lkMm1skhmSjFiGgp/gVj/Anbj1V1z7I6btLGzrgQuHc+5N7j1Bwpk2nvftFNbWNza3itvuzu7e/kHp8Kip41QR2iAxj1U7wJpyJmnDMMNpO1EUi4DTVjC6mfqtR6o0i+W9GSe0J/BAsogRbKzU7JJYo6d+qexVvRnQKvFzUoYc9X7ppxvGJBVUGsKx1h3fS0wvw8owwunE7aaaJpiM8IB2LJVYUN3LZttOUMUqIYpiZUsaNFP/TmRYaD0Wge0U2Az1sjcV//M6qYmuehmTSWqoJPOPopQjE6Pp6ShkihLDxwsPBmLiVkKGiWJ2eUSGWGFibGKuTcVfzmCVNM+rvuV3F+XadZ5PEU7gFM7Ah0uowS3UoQEEHuAFXuHNeXbenQ/nc95acPKZY1iA8/ULIZ+Vqg==
sin x AAAB/nicbVDLSgMxFL1TX3V8VV26CZaCqzIjgi6LblxWsA9oS8lkMm1skhmSjFiGgp/gVj/Anbj1V1z7I6bTLmzrgQuHc+5N7j1Bwpk2nvftFNbWNza3itvuzu7e/kHp8Kip41QR2iAxj1U7wJpyJmnDMMNpO1EUi4DTVjC6mfqtR6o0i+W9GSe0J/BAsogRbKzU7Gom0VO/VPaqXg60Svw5KdeKkKPeL/10w5ikgkpDONa643uJ6WVYGUY4nbjdVNMEkxEe0I6lEguqe1m+7QRVrBKiKFa2pEG5+nciw0LrsQhsp8BmqJe9qfif10lNdNXLmExSQyWZfRSlHJkYTU9HIVOUGD5eeDAQE7cSMkwUs8sjMsQKE2MTc20q/nIGq6R5XvUtv7so165n8UARTuAUzsCHS6jBLdShAQQe4AVe4c15dt6dD+dz1lpw5jPHsADn6xeQTJX3AAAB/nicbVDLSgMxFL1TX3V8VV26CZaCqzIjgi6LblxWsA9oS8lkMm1skhmSjFiGgp/gVj/Anbj1V1z7I6bTLmzrgQuHc+5N7j1Bwpk2nvftFNbWNza3itvuzu7e/kHp8Kip41QR2iAxj1U7wJpyJmnDMMNpO1EUi4DTVjC6mfqtR6o0i+W9GSe0J/BAsogRbKzU7Gom0VO/VPaqXg60Svw5KdeKkKPeL/10w5ikgkpDONa643uJ6WVYGUY4nbjdVNMEkxEe0I6lEguqe1m+7QRVrBKiKFa2pEG5+nciw0LrsQhsp8BmqJe9qfif10lNdNXLmExSQyWZfRSlHJkYTU9HIVOUGD5eeDAQE7cSMkwUs8sjMsQKE2MTc20q/nIGq6R5XvUtv7so165n8UARTuAUzsCHS6jBLdShAQQe4AVe4c15dt6dD+dz1lpw5jPHsADn6xeQTJX3AAAB/nicbVDLSgMxFL1TX3V8VV26CZaCqzIjgi6LblxWsA9oS8lkMm1skhmSjFiGgp/gVj/Anbj1V1z7I6bTLmzrgQuHc+5N7j1Bwpk2nvftFNbWNza3itvuzu7e/kHp8Kip41QR2iAxj1U7wJpyJmnDMMNpO1EUi4DTVjC6mfqtR6o0i+W9GSe0J/BAsogRbKzU7Gom0VO/VPaqXg60Svw5KdeKkKPeL/10w5ikgkpDONa643uJ6WVYGUY4nbjdVNMEkxEe0I6lEguqe1m+7QRVrBKiKFa2pEG5+nciw0LrsQhsp8BmqJe9qfif10lNdNXLmExSQyWZfRSlHJkYTU9HIVOUGD5eeDAQE7cSMkwUs8sjMsQKE2MTc20q/nIGq6R5XvUtv7so165n8UARTuAUzsCHS6jBLdShAQQe4AVe4c15dt6dD+dz1lpw5jPHsADn6xeQTJX3AAAB/nicbVDLSgMxFL1TX3V8VV26CZaCqzIjgi6LblxWsA9oS8lkMm1skhmSjFiGgp/gVj/Anbj1V1z7I6btLGzrgQuHc+5N7j1Bwpk2nvftFNbWNza3itvuzu7e/kHp8Kip41QR2iAxj1U7wJpyJmnDMMNpO1EUi4DTVjC6mfqtR6o0i+W9GSe0J/BAsogRbKzU7Gom0VO/VPaq3gxolfg5KUOOer/00w1jkgoqDeFY647vJaaXYWUY4XTidlNNE0xGeEA7lkosqO5ls20nqGKVEEWxsiUNmql/JzIstB6LwHYKbIZ62ZuK/3md1ERXvYzJJDVUkvlHUcqRidH0dBQyRYnh44UHAzFxKyHDRDG7PCJDrDAxNjHXpuIvZ7BKmudV3/K7i3LtOs+nCCdwCmfgwyXU4Bbq0AACD/ACr/DmPDvvzofzOW8tOPnMMSzA+foFKayVrw==
vectorization
0
π
4
π
2
3 π
4
π
0
π
4
π
2
3 π
4
π
cos x
AAAB/nicbVDLSgMxFL1TX3V8VV26CZaCqzIjgi6LblxWsA9oS8lkMm1skhmSjFiGgp/gVj/Anbj1V1z7I6bTLmzrgQuHc+5N7j1Bwpk2nvftFNbWNza3itvuzu7e/kHp8Kip41QR2iAxj1U7wJpyJmnDMMNpO1EUi4DTVjC6mfqtR6o0i+W9GSe0J/BAsogRbKzU7JJYo6d+qexVvRxolfhzUq4VIUe9X/rphjFJBZWGcKx1x/cS08uwMoxwOnG7qaYJJiM8oB1LJRZU97J82wmqWCVEUaxsSYNy9e9EhoXWYxHYToHNUC97U/E/r5Oa6KqXMZmkhkoy+yhKOTIxmp6OQqYoMXy88GAgJm4lZJgoZpdHZIgVJsYm5tpU/OUMVknzvOpbfndRrl3P4oEinMApnIEPl1CDW6hDAwg8wAu8wpvz7Lw7H87nrLXgzGeOYQHO1y+IP5XyAAAB/nicbVDLSgMxFL1TX3V8VV26CZaCqzIjgi6LblxWsA9oS8lkMm1skhmSjFiGgp/gVj/Anbj1V1z7I6bTLmzrgQuHc+5N7j1Bwpk2nvftFNbWNza3itvuzu7e/kHp8Kip41QR2iAxj1U7wJpyJmnDMMNpO1EUi4DTVjC6mfqtR6o0i+W9GSe0J/BAsogRbKzU7JJYo6d+qexVvRxolfhzUq4VIUe9X/rphjFJBZWGcKx1x/cS08uwMoxwOnG7qaYJJiM8oB1LJRZU97J82wmqWCVEUaxsSYNy9e9EhoXWYxHYToHNUC97U/E/r5Oa6KqXMZmkhkoy+yhKOTIxmp6OQqYoMXy88GAgJm4lZJgoZpdHZIgVJsYm5tpU/OUMVknzvOpbfndRrl3P4oEinMApnIEPl1CDW6hDAwg8wAu8wpvz7Lw7H87nrLXgzGeOYQHO1y+IP5XyAAAB/nicbVDLSgMxFL1TX3V8VV26CZaCqzIjgi6LblxWsA9oS8lkMm1skhmSjFiGgp/gVj/Anbj1V1z7I6bTLmzrgQuHc+5N7j1Bwpk2nvftFNbWNza3itvuzu7e/kHp8Kip41QR2iAxj1U7wJpyJmnDMMNpO1EUi4DTVjC6mfqtR6o0i+W9GSe0J/BAsogRbKzU7JJYo6d+qexVvRxolfhzUq4VIUe9X/rphjFJBZWGcKx1x/cS08uwMoxwOnG7qaYJJiM8oB1LJRZU97J82wmqWCVEUaxsSYNy9e9EhoXWYxHYToHNUC97U/E/r5Oa6KqXMZmkhkoy+yhKOTIxmp6OQqYoMXy88GAgJm4lZJgoZpdHZIgVJsYm5tpU/OUMVknzvOpbfndRrl3P4oEinMApnIEPl1CDW6hDAwg8wAu8wpvz7Lw7H87nrLXgzGeOYQHO1y+IP5XyAAAB/nicbVDLSgMxFL1TX3V8VV26CZaCqzIjgi6LblxWsA9oS8lkMm1skhmSjFiGgp/gVj/Anbj1V1z7I6btLGzrgQuHc+5N7j1Bwpk2nvftFNbWNza3itvuzu7e/kHp8Kip41QR2iAxj1U7wJpyJmnDMMNpO1EUi4DTVjC6mfqtR6o0i+W9GSe0J/BAsogRbKzU7JJYo6d+qexVvRnQKvFzUoYc9X7ppxvGJBVUGsKx1h3fS0wvw8owwunE7aaaJpiM8IB2LJVYUN3LZttOUMUqIYpiZUsaNFP/TmRYaD0Wge0U2Az1sjcV//M6qYmuehmTSWqoJPOPopQjE6Pp6ShkihLDxwsPBmLiVkKGiWJ2eUSGWGFibGKuTcVfzmCVNM+rvuV3F+XadZ5PEU7gFM7Ah0uowS3UoQEEHuAFXuHNeXbenQ/nc95acPKZY1iA8/ULIZ+Vqg==
sin x AAAB/nicbVDLSgMxFL1TX3V8VV26CZaCqzIjgi6LblxWsA9oS8lkMm1skhmSjFiGgp/gVj/Anbj1V1z7I6bTLmzrgQuHc+5N7j1Bwpk2nvftFNbWNza3itvuzu7e/kHp8Kip41QR2iAxj1U7wJpyJmnDMMNpO1EUi4DTVjC6mfqtR6o0i+W9GSe0J/BAsogRbKzU7Gom0VO/VPaqXg60Svw5KdeKkKPeL/10w5ikgkpDONa643uJ6WVYGUY4nbjdVNMEkxEe0I6lEguqe1m+7QRVrBKiKFa2pEG5+nciw0LrsQhsp8BmqJe9qfif10lNdNXLmExSQyWZfRSlHJkYTU9HIVOUGD5eeDAQE7cSMkwUs8sjMsQKE2MTc20q/nIGq6R5XvUtv7so165n8UARTuAUzsCHS6jBLdShAQQe4AVe4c15dt6dD+dz1lpw5jPHsADn6xeQTJX3AAAB/nicbVDLSgMxFL1TX3V8VV26CZaCqzIjgi6LblxWsA9oS8lkMm1skhmSjFiGgp/gVj/Anbj1V1z7I6bTLmzrgQuHc+5N7j1Bwpk2nvftFNbWNza3itvuzu7e/kHp8Kip41QR2iAxj1U7wJpyJmnDMMNpO1EUi4DTVjC6mfqtR6o0i+W9GSe0J/BAsogRbKzU7Gom0VO/VPaqXg60Svw5KdeKkKPeL/10w5ikgkpDONa643uJ6WVYGUY4nbjdVNMEkxEe0I6lEguqe1m+7QRVrBKiKFa2pEG5+nciw0LrsQhsp8BmqJe9qfif10lNdNXLmExSQyWZfRSlHJkYTU9HIVOUGD5eeDAQE7cSMkwUs8sjMsQKE2MTc20q/nIGq6R5XvUtv7so165n8UARTuAUzsCHS6jBLdShAQQe4AVe4c15dt6dD+dz1lpw5jPHsADn6xeQTJX3AAAB/nicbVDLSgMxFL1TX3V8VV26CZaCqzIjgi6LblxWsA9oS8lkMm1skhmSjFiGgp/gVj/Anbj1V1z7I6bTLmzrgQuHc+5N7j1Bwpk2nvftFNbWNza3itvuzu7e/kHp8Kip41QR2iAxj1U7wJpyJmnDMMNpO1EUi4DTVjC6mfqtR6o0i+W9GSe0J/BAsogRbKzU7Gom0VO/VPaqXg60Svw5KdeKkKPeL/10w5ikgkpDONa643uJ6WVYGUY4nbjdVNMEkxEe0I6lEguqe1m+7QRVrBKiKFa2pEG5+nciw0LrsQhsp8BmqJe9qfif10lNdNXLmExSQyWZfRSlHJkYTU9HIVOUGD5eeDAQE7cSMkwUs8sjMsQKE2MTc20q/nIGq6R5XvUtv7so165n8UARTuAUzsCHS6jBLdShAQQe4AVe4c15dt6dD+dz1lpw5jPHsADn6xeQTJX3AAAB/nicbVDLSgMxFL1TX3V8VV26CZaCqzIjgi6LblxWsA9oS8lkMm1skhmSjFiGgp/gVj/Anbj1V1z7I6btLGzrgQuHc+5N7j1Bwpk2nvftFNbWNza3itvuzu7e/kHp8Kip41QR2iAxj1U7wJpyJmnDMMNpO1EUi4DTVjC6mfqtR6o0i+W9GSe0J/BAsogRbKzU7Gom0VO/VPaq3gxolfg5KUOOer/00w1jkgoqDeFY647vJaaXYWUY4XTidlNNE0xGeEA7lkosqO5ls20nqGKVEEWxsiUNmql/JzIstB6LwHYKbIZ62ZuK/3md1ERXvYzJJDVUkvlHUcqRidH0dBQyRYnh44UHAzFxKyHDRDG7PCJDrDAxNjHXpuIvZ7BKmudV3/K7i3LtOs+nCCdwCmfgwyXU4Bbq0AACD/ACr/DmPDvvzofzOW8tOPnMMSzA+foFKayVrw==
⟨ f | g ⟩ =
b
a
f *( x ) g ( x ) dx
𝒏
$
, Hilbert space (vector space of infinite dimension)
&
- norm of 𝑓(𝑥) over interval [𝑎, 𝑏]
⟨sin x | cos x ⟩ =
π / 2
0
sin x cos xdx =
⟨sin x | cos x ⟩ =
π
0
sin x cos xdx = 0
∥ sin x ∥ =
∫
π / 2
0
sin x
2
dx =
π
4
∥ sin x ∥ =
∫
π
0
sin x
2
dx =
π
2
∥ f ∥ =
b
a
| f |
2
d x
Least-Square of continuous function
c
1
f
1
( x ) + c
2
f
2
( x ) = g ( x )
⋮ ⋮
| |
f
1
( x
i
) f
2
( x
i
)
| |
⋮ ⋮
[
c
1
c
2
]
=
⋮
|
g ( x
i
)
|
⋮
Minimize ∫
7
8
=
=
&
&
&
𝒎×𝟐
𝒎×𝟏
[
c
1
c
2
]
= ( A
T
A )
− 1
2 × 2
( A
T
b )
2 × 1
Orthonormal basis on [𝑎, 𝑏]: hf
i
|f
j
i =
Z
b
a
dxf
⇤
i
(x)f
j
(x) =
ij
AAACOHicbVDLbtQwFL1peQzDayhLNhajSoXFKCmVyqZoBBuWReq0lSbT6Ma5mbp1nMi+QR2F+Ru+gk9gCxt2SCwQW76gnkwXfR3J1vE5917bJ620chyGv4KV1Tt3793vPOg+fPT4ydPes7V9V9ZW0kiWurSHKTrSytCIFWs6rCxhkWo6SE8/LPyDz2SdKs0ezyqaFDg1KlcS2UtJ712s0Uw1iTxRX/LkRMS2Pe/EynCCR6nIzhbe0euNs1fe9/tOnJFmTBp1Mk96/XAQthA3SXRB+sMOtNhNen/irJR1QYalRufGUVjxpEHLSmqad+PaUYXyFKc09tRgQW7StP+ci3WvZCIvrV+GRate7miwcG5WpL6yQD52172FeJs3rjl/O2mUqWomI5cX5bUWXIpFaCJTliTr2ZWBaTHvrmcKpVX+8UIeo0XJPuuuTyW6nsFNsr85iN4MNj9t9Yfvl/FAB17AS9iACLZhCB9hF0Yg4St8hx/wM/gW/A7+Bv+WpSvBRc9zuILg/znxzKyf
c i
= hf i
|gi =
Z
b
a
dxf
⇤
i
(x)g(x)
AAACK3icbVDLSgMxFL1TX7W+qi7dBIugLsqMCropFN24rGCr0GmHTCYzhmYyQ5IRS+13+BV+glv9AFeKO/E/TKddaPXAhZNz7r1Jjp9yprRtv1mFmdm5+YXiYmlpeWV1rby+0VJJJgltkoQn8trHinImaFMzzel1KimOfU6v/N7ZyL+6pVKxRFzqfko7MY4ECxnB2khe2SEeq7kci4hTFHrsPnJlfqi5TGgPd30U3I2M7v7u3V5kyitX7KqdA/0lzoRU6kXI0fDKn26QkCymQhOOlWo7dqo7Ayw1I5wOS26maIpJD0e0bajAMVWdQf61IdoxSoDCRJoSGuXqz4kBjpXqx77pjLG+UdPeSPzPa2c6POkMmEgzTQUZXxRmHOkEjXJCAZOUaN7/tdCPh6WdgGEimXk8IjdYYqJNvCWTijOdwV/SOqg6h9WDi6NK/XQcDxRhC7ZhFxw4hjqcQwOaQOABnuAZXqxH69V6tz7GrQVrMrMJv2B9fQMD6KcI
i = 1 or 2 AAACCXicbVDLSsNAFL2prxofrbp0M1gKrkpSBd0IRTcuK9gHtKFMJpN26EwmzEzEEvIFfoJb/QB34tavcO2PmD4W2npWh3Puvedy/JgzbRznyyqsrW9sbhW37Z3dvf1S+eCwrWWiCG0RyaXq+lhTziLaMsxw2o0VxcLntOOPb6Z+54EqzWR0byYx9QQeRixkBJtcGpRL7MrtC18+pkgqlNUH5YpTc2ZAq8RdkEqjCDM0B+XvfiBJImhkCMda91wnNl6KlWGE08zuJ5rGmIzxkPZyGmFBtZfOHs9QNVcCFObJoYwMmqm/N1IstJ4IP58U2Iz0sjcV//N6iQkvvZRFcWJoROZBYcKRkWjaAgqYosTwyZ+DvsjsasAwUSx/HpERVpiYvDw7b8Vd7mCVtOs196xWvzuvNK7n9UARjuEETsGFC2jALTShBQQSeIYXeLWerDfr3fqYjxasxc4R/IH1+QMnP5l8
Least-Square Data fitting
b ( t ) = Cf
1
( t ) + Df
2
( t )
[1, 1 , · · · 1]
AAACBnicbVDLSsNAFL3xWeOr6tLNYCm4KCURQZdFNy4r2AeksUwmk3boJBNmJkIJ3fsJbvUD3Ilbf8O1P+I07cK2HrhwOOe+OEHKmdKO822trW9sbm2Xduzdvf2Dw/LRcVuJTBLaIoIL2Q2wopwltKWZ5rSbSorjgNNOMLqd+p0nKhUTyYMep9SP8SBhESNYG+nRc2turUdCoRVy/X654tSdAmiVuHNSaZSgQLNf/umFgmQxTTThWCnPdVLt51hqRjid2L1M0RSTER5Qz9AEx1T5efH1BFWNEqJISFOJRoX6dyLHsVLjODCdMdZDtexNxf88L9PRtZ+zJM00TcjsUJRxpAWaRoBCJinRfLywMIgndjVkmEhmnkdkiCUm2iRnm1Tc5QxWSfui7hp+f1lp3MzigRKcwhmcgwtX0IA7aEILCEh4gVd4s56td+vD+py1rlnzmRNYgPX1C/O2mEU=AAACBnicbVDLSsNAFL3xWeOr6tLNYCm4KCURQZdFNy4r2AeksUwmk3boJBNmJkIJ3fsJbvUD3Ilbf8O1P+I07cK2HrhwOOe+OEHKmdKO822trW9sbm2Xduzdvf2Dw/LRcVuJTBLaIoIL2Q2wopwltKWZ5rSbSorjgNNOMLqd+p0nKhUTyYMep9SP8SBhESNYG+nRc2turUdCoRVy/X654tSdAmiVuHNSaZSgQLNf/umFgmQxTTThWCnPdVLt51hqRjid2L1M0RSTER5Qz9AEx1T5efH1BFWNEqJISFOJRoX6dyLHsVLjODCdMdZDtexNxf88L9PRtZ+zJM00TcjsUJRxpAWaRoBCJinRfLywMIgndjVkmEhmnkdkiCUm2iRnm1Tc5QxWSfui7hp+f1lp3MzigRKcwhmcgwtX0IA7aEILCEh4gVd4s56td+vD+py1rlnzmRNYgPX1C/O2mEU=AAACBnicbVDLSsNAFL3xWeOr6tLNYCm4KCURQZdFNy4r2AeksUwmk3boJBNmJkIJ3fsJbvUD3Ilbf8O1P+I07cK2HrhwOOe+OEHKmdKO822trW9sbm2Xduzdvf2Dw/LRcVuJTBLaIoIL2Q2wopwltKWZ5rSbSorjgNNOMLqd+p0nKhUTyYMep9SP8SBhESNYG+nRc2turUdCoRVy/X654tSdAmiVuHNSaZSgQLNf/umFgmQxTTThWCnPdVLt51hqRjid2L1M0RSTER5Qz9AEx1T5efH1BFWNEqJISFOJRoX6dyLHsVLjODCdMdZDtexNxf88L9PRtZ+zJM00TcjsUJRxpAWaRoBCJinRfLywMIgndjVkmEhmnkdkiCUm2iRnm1Tc5QxWSfui7hp+f1lp3MzigRKcwhmcgwtX0IA7aEILCEh4gVd4s56td+vD+py1rlnzmRNYgPX1C/O2mEU=AAACBnicbVDLSgMxFM34rOOr6tJNsBRclDIRQZdFNy4r2AdMx5LJZNrQZDIkGaEM3fsJbvUD3Ilbf8O1P2LazsK2HrhwOOe+OGHKmTae9+2srW9sbm2Xdtzdvf2Dw/LRcVvLTBHaIpJL1Q2xppwltGWY4bSbKopFyGknHN1O/c4TVZrJ5MGMUxoIPEhYzAg2Vnr0UQ3VeiSSRkMU9MsVr+7NAFcJKkgFFGj2yz+9SJJM0MQQjrX2kZeaIMfKMMLpxO1lmqaYjPCA+pYmWFAd5LOvJ7BqlQjGUtlKDJypfydyLLQei9B2CmyGetmbiv95fmbi6yBnSZoZmpD5oTjj0Eg4jQBGTFFi+HhhYSgmbjVimChmn4dkiBUmxibn2lTQcgarpH1RR5bfX1YaN0U+JXAKzsA5QOAKNMAdaIIWIECBF/AK3pxn5935cD7nrWtOMXMCFuB8/QKNFpf9
[t 1
, t 2
, · · · t m
]
AAACDnicbVDLSsNAFL2pr1pf9bFzM1gEF6UkRdBl0Y1LBWsLbQiTybQdOpOEmRuhlv6Dn+BWP8CduPUXXPsjTtMu1HrgwuGc++KEqRQGXffTKSwtr6yuFddLG5tb2zvl3b07k2Sa8SZLZKLbITVcipg3UaDk7VRzqkLJW+Hwcuq37rk2IolvcZRyX9F+LHqCUbRSUD7oYOBVMahXuyxK0BAMlB+UK27NzUEWiTcnlUYRclwH5a9ulLBM8RiZpMZ0PDdFf0w1Cib5pNTNDE8pG9I+71gaU8WNP86/n5Bjq0Skl2hbMZJc/TkxpsqYkQptp6I4MH+9qfif18mwd+6PRZxmyGM2O9TLJMGETKMgkdCcoRz9WhiqSek4EpRpYZ8nbEA1ZWgTLNlUvL8ZLJK7es2z/Oa00riYxQNFOIQjOAEPzqABV3ANTWDwAE/wDC/Oo/PqvDnvs9aCM5/Zh19wPr4BvFubaA==AAACDnicbVDLSsNAFL2pr1pf9bFzM1gEF6UkRdBl0Y1LBWsLbQiTybQdOpOEmRuhlv6Dn+BWP8CduPUXXPsjTtMu1HrgwuGc++KEqRQGXffTKSwtr6yuFddLG5tb2zvl3b07k2Sa8SZLZKLbITVcipg3UaDk7VRzqkLJW+Hwcuq37rk2IolvcZRyX9F+LHqCUbRSUD7oYOBVMahXuyxK0BAMlB+UK27NzUEWiTcnlUYRclwH5a9ulLBM8RiZpMZ0PDdFf0w1Cib5pNTNDE8pG9I+71gaU8WNP86/n5Bjq0Skl2hbMZJc/TkxpsqYkQptp6I4MH+9qfif18mwd+6PRZxmyGM2O9TLJMGETKMgkdCcoRz9WhiqSek4EpRpYZ8nbEA1ZWgTLNlUvL8ZLJK7es2z/Oa00riYxQNFOIQjOAEPzqABV3ANTWDwAE/wDC/Oo/PqvDnvs9aCM5/Zh19wPr4BvFubaA==AAACDnicbVDLSsNAFL2pr1pf9bFzM1gEF6UkRdBl0Y1LBWsLbQiTybQdOpOEmRuhlv6Dn+BWP8CduPUXXPsjTtMu1HrgwuGc++KEqRQGXffTKSwtr6yuFddLG5tb2zvl3b07k2Sa8SZLZKLbITVcipg3UaDk7VRzqkLJW+Hwcuq37rk2IolvcZRyX9F+LHqCUbRSUD7oYOBVMahXuyxK0BAMlB+UK27NzUEWiTcnlUYRclwH5a9ulLBM8RiZpMZ0PDdFf0w1Cib5pNTNDE8pG9I+71gaU8WNP86/n5Bjq0Skl2hbMZJc/TkxpsqYkQptp6I4MH+9qfif18mwd+6PRZxmyGM2O9TLJMGETKMgkdCcoRz9WhiqSek4EpRpYZ8nbEA1ZWgTLNlUvL8ZLJK7es2z/Oa00riYxQNFOIQjOAEPzqABV3ANTWDwAE/wDC/Oo/PqvDnvs9aCM5/Zh19wPr4BvFubaA==AAACDnicbVDLSsNAFJ3UV42v+Ni5GSwFF6UkRdBl0Y3LCvYBbQiTybQdOpOEmRuhlv6Dn+BWP8CduPUXXPsjTtsstPXAhcM598UJU8E1uO6XVVhb39jcKm7bO7t7+wfO4VFLJ5mirEkTkahOSDQTPGZN4CBYJ1WMyFCwdji6mfntB6Y0T+J7GKfMl2QQ8z6nBIwUOCddCLwKBLVKj0YJaAyB9AOn5FbdOfAq8XJSQjkagfPdixKaSRYDFUTrruem4E+IAk4Fm9q9TLOU0BEZsK6hMZFM+5P591NcNkqE+4kyFQOeq78nJkRqPZah6ZQEhnrZm4n/ed0M+lf+hMdpBiymi0P9TGBI8CwKHHHFKIjxn4WhnNrliBOquHke0yFRhIJJ0DapeMsZrJJWreoZfndRql/n+RTRKTpD58hDl6iOblEDNRFFj+gZvaBX68l6s96tj0VrwcpnjtEfWJ8/VbubIA==
& subspace of
I
[b 1
, b 2
, · · · b m
]
AAACDnicbVDLSgMxFL1TX7W+6mPnJlgKLkqZKYIui25cVrAPaIchyaRtaDIzJBmhlv6Dn+BWP8CduPUXXPsjptMutPXAhcM598UhieDauO6Xk1tb39jcym8Xdnb39g+Kh0ctHaeKsiaNRaw6BGsmeMSahhvBOoliWBLB2mR0M/PbD0xpHkf3ZpwwX+JBxPucYmOloHjSJYFXIUGt0qNhbDQigfSDYsmtuhnQKvEWpFTPQ4ZGUPzuhTFNJYsMFVjrrucmxp9gZTgVbFropZolmI7wgHUtjbBk2p9k309R2Soh6sfKVmRQpv6emGCp9VgS2ymxGeplbyb+53VT07/yJzxKUsMiOj/UTwUyMZpFgUKuGDVi/GchkdNCOeSYKm6fR3SIFabGJliwqXjLGaySVq3qWX53Uapfz+OBPJzCGZyDB5dQh1toQBMoPMIzvMCr8+S8Oe/Ox7w15yxmjuEPnM8fZHebMg==AAACDnicbVDLSgMxFL1TX7W+6mPnJlgKLkqZKYIui25cVrAPaIchyaRtaDIzJBmhlv6Dn+BWP8CduPUXXPsjptMutPXAhcM598UhieDauO6Xk1tb39jcym8Xdnb39g+Kh0ctHaeKsiaNRaw6BGsmeMSahhvBOoliWBLB2mR0M/PbD0xpHkf3ZpwwX+JBxPucYmOloHjSJYFXIUGt0qNhbDQigfSDYsmtuhnQKvEWpFTPQ4ZGUPzuhTFNJYsMFVjrrucmxp9gZTgVbFropZolmI7wgHUtjbBk2p9k309R2Soh6sfKVmRQpv6emGCp9VgS2ymxGeplbyb+53VT07/yJzxKUsMiOj/UTwUyMZpFgUKuGDVi/GchkdNCOeSYKm6fR3SIFabGJliwqXjLGaySVq3qWX53Uapfz+OBPJzCGZyDB5dQh1toQBMoPMIzvMCr8+S8Oe/Ox7w15yxmjuEPnM8fZHebMg==AAACDnicbVDLSgMxFL1TX7W+6mPnJlgKLkqZKYIui25cVrAPaIchyaRtaDIzJBmhlv6Dn+BWP8CduPUXXPsjptMutPXAhcM598UhieDauO6Xk1tb39jcym8Xdnb39g+Kh0ctHaeKsiaNRaw6BGsmeMSahhvBOoliWBLB2mR0M/PbD0xpHkf3ZpwwX+JBxPucYmOloHjSJYFXIUGt0qNhbDQigfSDYsmtuhnQKvEWpFTPQ4ZGUPzuhTFNJYsMFVjrrucmxp9gZTgVbFropZolmI7wgHUtjbBk2p9k309R2Soh6sfKVmRQpv6emGCp9VgS2ymxGeplbyb+53VT07/yJzxKUsMiOj/UTwUyMZpFgUKuGDVi/GchkdNCOeSYKm6fR3SIFabGJliwqXjLGaySVq3qWX53Uapfz+OBPJzCGZyDB5dQh1toQBMoPMIzvMCr8+S8Oe/Ox7w15yxmjuEPnM8fZHebMg==AAACDnicbVDLSsNAFJ3UV42v+Ni5GSwFF6UkRdBl0Y3LCvYBbQgzk0k7dCYJMxOhhv6Dn+BWP8CduPUXXPsjTtsstPXAhcM598XBKWdKu+6XVVpb39jcKm/bO7t7+wfO4VFHJZkktE0SnsgeRopyFtO2ZprTXiopEpjTLh7fzPzuA5WKJfG9nqTUF2gYs4gRpI0UOCd9HHg1HDRqAxImWkEcCD9wKm7dnQOuEq8gFVCgFTjfgzAhmaCxJhwp1ffcVPs5kpoRTqf2IFM0RWSMhrRvaIwEVX4+/34Kq0YJYZRIU7GGc/X3RI6EUhOBTadAeqSWvZn4n9fPdHTl5yxOM01jsjgUZRzqBM6igCGTlGg++bMQi6ldDRkikpnnIRkhiYg2CdomFW85g1XSadQ9w+8uKs3rIp8yOAVn4Bx44BI0wS1ogTYg4BE8gxfwaj1Zb9a79bFoLVnFzDH4A+vzB/3Imuo=
J
J
A =
A
T
A =
[
]
y =
A
T
y =
[
]
Example: Fit the following data into a line: 𝑦 = 𝐶 + 𝐷 𝑡
1 2 3 4
2
4
6
8
10
12
A
T
A
[
C
D
]
= A
T
y
C = − 2, D = 3.
y = − 2 + 3.1 t
Least-Square Data fitting
1 2 3 4
2
4
6
8
10
12
y = − 2 + 3.1 t
Py
P = A ( A
T
A )
− 1
A
T
Projection operator:
𝐶 𝐴
Py =
J
J
Example: Fit the following data into a line: 𝑦 = 𝐶 + 𝐷 𝑡
Least-Square Data fitting
A
1
left AAACC3icbVDLSsNAFL3xWesr1aWbwVJwY0mqoMuqG5cV7APaGCbTSTt0JgkzE6WEfIKf4FY/wJ249SNc+yOmaRe29cCFwzn3xfEizpS2rG9jZXVtfWOzsFXc3tnd2zdLBy0VxpLQJgl5KDseVpSzgDY105x2Ikmx8Dhte6Obid9+pFKxMLjX44g6Ag8C5jOCdSa5ZunKTXpSIE59nT4kp3bqmmWrauVAy8SekXK9ADkarvnT64ckFjTQhGOlurYVaSfBUjPCaVrsxYpGmIzwgHYzGmBBlZPkr6eokil95Icyq0CjXP07kWCh1Fh4WafAeqgWvYn4n9eNtX/pJCyIYk0DMj3kxxzpEE1yQH0mKdF8PLfQE2mx0meYSJY9j8gQS0x0Fl8xS8VezGCZtGpV+6xauzsv16+n8UABjuAYTsCGC6jDLTSgCQSe4AVe4c14Nt6ND+Nz2rpizGYOYQ7G1y9EApqu
J
J
Example: Fit the following data into a curve: 𝑦 = 𝑎 + 𝑏 𝑡 + 𝑐 𝑡
&
P
1 2 3 4
4
6
8
10
12
14
A =
A
T
A
[
a
b
c
]
= A
T
y
y = 1.75 − 0.65 t + 0.75 t
2
1 2 3 4
4
6
8
10
12
y = 7 − 9 t + 4.5 t
2
− 0.5 t
3
A
T
A
a
b
c
d
= A
T
y A =
Least-Square Data fitting
1 2 3 4
2
4
6
8
10
12
J
J
Example: Fit the following data into a line: 𝑦 = 𝑎 + 𝑏 𝑡
WAx = Wy
W =
2
2
1
0
Weighting function:
( WA )
T
( WA ) = ( WA )
T
Wy
A
T
( W
T
W ) A = A ( W
T
W ) y
A
T
GA = AGy G = W
T
W
⟨ x | y ⟩ = x
T
Gy
Metric tensor
∥ x ∥ = ⟨ x | x ⟩ = x
T
Gx
y = −
17
9
8
3
t
Reweighting
errors!
More credible
measurements
Least-Square Data fitting
Example:
§ 2D Rotation matrix
§ 2D Projection onto 𝜃-line
§ 2D Mirror reflection (Householder)
§ Pauli matrices
§ Permutation matrices
g
cos 𝜃 − sin 𝜃
sin 𝜃 cos 𝜃
cos
&
𝜃 cos 𝜃 sin 𝜃
cos 𝜃 sin 𝜃 sin
&
cos 2 𝜃 sin 2𝜃
sin 2 𝜃 − cos 2 𝜃
=
&
P
When 𝑈
d
𝑈 = 𝐼, we have:
d
is the left-inverse of 𝑈 when 𝑈 is rectangular (𝑚 > 𝑛 full-column rank)
§ 𝑈𝑥 = 𝑥 preserving norm thus “unitary”
d
Q=
if 𝑈 is invertible (𝑚 = 𝑛 full rank)
d
d
= 𝐼 rows and columns are both orthonormal
§ Decomposition | b ⟩ = x
1
| q
1
⟩ + x
2
| q
2
⟩ + ⋯ + x
n
| q
n
I deserve your
memories
Still remember Fourier?
Parseval’s theorem
= | q
1
⟩⟨ q
1
| b ⟩ + | q
2
⟩⟨ q
2
| b ⟩ + ⋯ + | q
n
⟩⟨ q
n
| b ⟩
||b||
2
= |x 1
|
2
|
2
|
2
AAACIHicbVDLTgIxFL2DL8QX6tLNRIIxMSEzaKIbE6Ibl5jIIwEknU6Bhk5n0naMZIY/8Cv8BLf6Ae6MS936I3YGFgLepDnnnvtoe5yAUaks68vILC2vrK5l13Mbm1vbO/ndvbr0Q4FJDfvMF00HScIoJzVFFSPNQBDkOYw0nOF1Um88ECGpz+/UKCAdD/U57VGMlJa6+aM4duL4vnwZP3ZtjScaywm2sesrmaRcp918wSpZaZiLxJ6SQiULaVS7+Z+26+PQI1xhhqRs2VagOhESimJGxrl2KEmA8BD1SUtTjjwiO1H6n7FZ1Ipr9nyhD1dmqv6diJAn5chzdKeH1EDO1xLxv1orVL2LTkR5ECrC8eSiXshM5ZuJOaZLBcGKjWYWOt44V3QpwoLqx5t4gATCSnua067Y8x4sknq5ZJ+WyrdnhcrVxB7IwgEcwjHYcA4VuIEq1ADDE7zAK7wZz8a78WF8TlozxnRmH2bC+P4FWaOjNQ==
When 𝑈
d
𝑈 = 𝐼, we have:
§ Simplified least-square solution
§ Simplified projection operator
P = Q ( Q
T
Q )
− 1
Q
T
= QQ
T
Q
T
Qx = Q
T
b x = Q
T
b
c =
m
m
i = 1
y
i
= y ¯ d =
m
i = 1
( t
i
t ) y
i
m
i = 1
( t
i
t )
2
[
m 0
i
( t
i
t )
2
]
[
c
d
]
[
i
( y
i
− y ¯)
i
( t
i
t )( y
i
− y ¯) [ ]
m ∑
i
t
i
i
t
i
i
t
2
i
]
[
C
D
]
[
i
y
i
i
t
i
y
i
]
Example: Least-square data fitting
