Python Numeric Cook List
Alguns macetes na utilização do python para resolução de problemas numéricos/de engenharia/científicos:
Receitas numéricas
Equações diferenciais
- ode - ordinary differential equation solver: http://help.scilab.org/docs/5.3.2/en_US/ode.html
- Resolução de sistemas de equações diferenciais ordinárias [ode]: http://docs.scipy.org/doc/scipy/reference/tutorial/integrate.html
- Modeling with ordinary differential equations (ODEs): http://lister.dulci.duhs.duke.edu/~cliburn/summer-school/python/_build/html/ode_models.html
FFT
- Receita de bolo para a FFT http://www.cbcity.de-/die-fft-mit-python-einfach-erklaert
- Fourier Transforms (scipy.fftpack) http://docs.scipy.org/doc/scipy-0.14.0/reference/tutorial/fftpack.html
- FFT na placa de vídeo: https://pythonhosted.org/pyfft/
Filtros
- Bibliotecas de processamento de sinais scipy: http://docs.scipy.org/doc/scipy-0.14.0/reference/signal.html
- Rotina para construção de filtro Chebyschev Tipo II : http://docs.scipy.org/doc/scipy/reference/generated/scipy.signal.cheby2.html#scipy.signal.cheby2
- Rotina para construção de filtro Butterworth: http://docs.scipy.org/doc/scipy/reference/generated/scipy.signal.butter.html
- Construindo um filtro passa-banda Butterworth: http://wiki.scipy.org/Cookbook/ButterworthBandpass
- Levantar resposta em frequência de um filtro analógico: http://docs.scipy.org/doc/scipy/reference/generated/scipy.signal.freqs.html#scipy.signal.freqs
- Levantar resposta em frequência de um filtro digital: http://docs.scipy.org/doc/scipy/reference/generated/scipy.signal.freqz.html
- Aplicar um filtro desenvolvido em forward E backward (preserva fase): http://docs.scipy.org/doc/scipy-0.14.0/reference/generated/scipy.signal.filtfilt.html
- Aplicar um filtro desenvolvido em forward OU backward: http://docs.scipy.org/doc/scipy/reference/generated/scipy.signal.lfilter.html
Matemática simbólica
- Solving symbolic equations with sympy http://docs.sympy.org/dev/modules/solvers/solvers.html
Otimização
- Resumos dos métodos de otimização: http://scipy-lectures.github.io/advanced/mathematical_optimization/ (Se você não é familiarizado com métodos de otimização - Esta é uma ótima primeira leitura!)
- Optimization (scipy.optimize) - Tutorial: http://docs.scipy.org/doc/scipy/reference/tutorial/optimize.html
- Todas as rotinas de otimização do scipy: http://docs.scipy.org/doc/scipy/reference/optimize.html
Álgebra
- Calculando o rank e o nullspace: http://scipy-cookbook.readthedocs.io/items/RankNullspace.html
Numpy+Scipy
- Lendo dados de arquivo texto direto para matriz numpy http://docs.scipy.org/doc/numpy/reference/generated/numpy.loadtxt.html
- Rosetta stone Matlab x Numpy: http://mathesaurus.sourceforge.net/matlab-numpy.html
- Mais uma ótima tabela comparativa da manipulação de matrizes Numpy x Matlab http://sebastianraschka.com/Articles/2014_matlab_vs_numpy.html
- Slicing, Indexing, Filtering and selecting data from arrays:
Plotando gráficos
- Tutorial básico do uso do pylab: http://jakevdp.github.io/mpl_tutorial/index.html
- Página oficial do matplotlib: http://matplotlib.org/
- Guia do usuário do matplotlib: http://matplotlib.org/users/index.html
- Convertendo texto utf-8 para unicode (pau do matplotlib): http://stackoverflow.com/questions/2406700/accented-characters-in-matplotlib/2406765#2406765
- Writing mathematical expressions http://matplotlib.org/users/mathtext.html
- Plotando vetores: http://stackoverflow.com/questions/12265234/plotting-2d-vectors-in-python-matplotlib
- Resolução de alguns problemas associados aos terminais de plotagem interativa e seleção dos mesmos https://daemoniolabs.wordpress.com/2014/12/09/instalando-pylab-matplotlib-no-opensuse-13-2/
- Algumas opções para visualização de gráficos além da matplotlib http://pbpython.com/visualization-tools-1.html
- Estilos para serem utilizados com a matplotlib http://matplotlib.org/examples/style_sheets/index.html
- Plotando múltiplas figuras e mantendo a memória sã: usar plt.clf(); plt.close(figure)
- Plotando múltiplas curvas no mesmo gráfico com degradée (gradiente): http://stackoverflow.com/questions/4971269/how-to-pick-a-new-color-for-each-plotted-line-within-a-figure-in-matplotlib
- Remover bordas da legenda e tornar a caixa transparente: http://stackoverflow.com/questions/25540259/remove-or-adapt-border-of-frame-of-legend-using-matplotlib
- Ajustando os "majors" e "minors" xticks e yticks: http://stackoverflow.com/questions/24943991/matplotlib-change-grid-interval-and-specify-tick-larocandobels
- Mudando automaticamente os tipos de linhas: http://stackoverflow.com/questions/7799156/can-i-cycle-through-line-styles-in-matplotlib
- Trocando o lado do eixo-y http://stackoverflow.com/questions/10354397/python-matplotlib-y-axis-ticks-on-right-side-of-plot
- Colorbar: http://stackoverflow.com/questions/14777066/matplotlib-discrete-colorbar
Mixing Python com outras linguagens
- Usando C e Fortran com Python (mantendo compatibilidade com os tipos do SciPy): http://nbviewer.ipython.org/urls/raw.github.com/jrjohansson/scientific-python-lectures/master/Lecture-6A-Fortran-and-C.ipynb
- f2py (Fortran e Python):
- Tutorial f2py (desenvolva rápido com python, execute rápido com fortran): http://cens.ioc.ee/projects/f2py2e/usersguide/
- Ótimo material da Universidade de Cambridge para integrar o Fortran com o Python: http://www.ucs.cam.ac.uk/docs/course-notes/unix-courses/pythonfortran/files/f2py.pdf
- F2PY Users Guide and Reference Manual: http://docs.scipy.org/doc/numpy-dev/f2py/
- Python x Fortran 90 Rosetta stone: http://www.fortran90.org/src/rosetta.html
- Introduction to Jython -- A vantagem do Pyhton (linguagem) combinada com a vantagem do Java (ambiente): http://www.jython.org/jythonbook/en/1.0/
- Python x Julia:
- https://juliabyexample.helpmanual.io/
- http://blog.leahhanson.us/post/julia/julia-calling-python.html
- https://github.com/JuliaPy/PyCall.jl
- https://github.com/JuliaPy/pyjulia
- https://groups.google.com/forum/#!topic/julia-users/oAWYShR1Pks - from julia import XXX
- https://github.com/JuliaPy/pyjulia/issues/39 - j.using("moduleName")
Processamento paralelo
- Diversas API, módulos, técnicas, etc: https://wiki.python.org/moin/ParallelProcessing
- Processamento paralelo em 1 (UMA) linha!!!: https://medium.com/building-things-on-the-internet/40e9b2b36148
- Uso de Queues para retornar dados das threads: http://www.troyfawkes.com/learn-python-multithreading-queues-basics/
- iPython paralelo: http://research-it.wharton.upenn.edu/news/parallel-ipython-with-univa-grid-engine-sge/
- Parallel Programming with numpy and scipy: http://scipy.github.io/old-wiki/pages/ParallelProgramming
- Numba is an open-source NumPy-aware optimizing compiler for Python: https://numba.pydata.org/
- Installation of conda (numba package manager): https://conda.io/miniconda.html
- GPU:
- Python + GPU NVIDIA (PyCUDA): http://mathema.tician.de/software/pycuda
- scikit-cuda, substituto do scikit na GPU: https://scikit-cuda.readthedocs.io/en/latest/
- Theano, python library for math processing: http://deeplearning.net/software/theano/introduction.html#
- Dask: provides advanced parallelism for analytics, enabling performance at scale for the tools you love https://dask.org/
- Artigos variados:
- First impressions of GPUs and PyData: http://matthewrocklin.com/blog/work/2018/12/17/gpu-python-challenges
Data Science
- Pandas
- Tutorials: https://pandas.pydata.org/pandas-docs/stable/tutorials.html
- Tutorial Panda Datacamp: https://www.datacamp.com/community/tutorials/pandas-tutorial-dataframe-python
- Filtro por data em Pandas: https://stackoverflow.com/questions/29370057/select-dataframe-rows-between-two-dates
- Removendo horas de uma data: https://stackoverflow.com/questions/24786209/dropping-time-from-datetime-m8-in-pandas
- Buscando valores únicos em Pandas: https://chrisalbon.com/python/data_wrangling/pandas_list_unique_values_in_column/
- Encontrar NaN: https://stackoverflow.com/questions/14247586/python-pandas-how-to-select-rows-with-one-or-more-nulls-from-a-dataframe-without
- Remover linhas com NaN: https://stackoverflow.com/questions/13413590/how-to-drop-rows-of-pandas-dataframe-whose-value-in-certain-columns-is-nan
- Informações acerca do dataframe: https://pandas.pydata.org/pandas-docs/stable/generated/pandas.DataFrame.info.html
- groupby, aggregation and new DataFrame:
- http://pandas.pydata.org/pandas-docs/stable/generated/pandas.core.groupby.DataFrameGroupBy.agg.html
- https://www.shanelynn.ie/summarising-aggregation-and-grouping-data-in-python-pandas/
- https://stackoverflow.com/questions/10373660/converting-a-pandas-groupby-object-to-dataframe
- https://stackoverflow.com/questions/43552105/using-pandas-groupby-to-create-new-dataframe-containing-all-columns-of-parent-da
- https://stackoverflow.com/questions/16266019/python-pandas-group-datetime-column-into-hour-and-minute-aggregations
- https://pandas.pydata.org/pandas-docs/stable/generated/pandas.DataFrame.groupby.html
- Ajustando o tipo de dados da coluna https://stackoverflow.com/questions/15125343/how-to-iterate-through-two-pandas-columns
- http://www.ritchieng.com/pandas-changing-datatype/
- https://pandas.pydata.org/pandas-docs/stable/generated/pandas.DataFrame.astype.html
- https://stackoverflow.com/questions/30121181/how-to-convert-string-into-float-value-in-the-dataframe
- Iterar sobre múltiplas colunas: https://stackoverflow.com/questions/15125343/how-to-iterate-through-two-pandas-columns
- Criando novas colunas baseadas nas demais: https://chrisalbon.com/python/data_wrangling/pandas_create_column_using_conditional/
- 19 exemplos de manipulação de dados com pandas: https://jeffdelaney.me/blog/useful-snippets-in-pandas/
- Alterar o valor de uma célula específica: https://stackoverflow.com/questions/13842088/set-value-for-particular-cell-in-pandas-dataframe
- A Beginner’s Guide to Optimizing Pandas Code for Speed: https://engineering.upside.com/a-beginners-guide-to-optimizing-pandas-code-for-speed-c09ef2c6a4d6
- Interseção de dois dataframes: https://stackoverflow.com/questions/26921943/pandas-intersection-of-two-data-frames-based-on-column-entries
- Lendo arquivos json: https://hayd.github.io/2013/pandas-json
- Lendo arquivos com FIXED COLUMN WITDH https://www.programcreek.com/python/example/101362/pandas.read_fwf https://pandas.pydata.org/pandas-docs/version/0.20/generated/pandas.read_fwf.html
- Expandindo resolução de tempo: https://stackoverflow.com/questions/13726573/create-a-24-hour-1-min-resolution-data-set-in-pandas
- TensorFlow vs. scikit-learn: The Microbiome Challenge: http://alifar76.github.io/sklearn-metrics/
- Azure Studio Machine Learning: https://studio.azureml.net
- Ferramentas que podem ser utilizadas para trabalhar com grande conjuntos de dados: numpy, h5py, databases, HDF5 support, mpi4py, dask, blaze: https://datascience.stackexchange.com/questions/778/is-python-suitable-for-big-data/796
- Scikit:
- Decision Trees: http://scikit-learn.org/stable/modules/tree.html
- Decision Trees Regression: http://scikit-learn.org/stable/auto_examples/tree/plot_tree_regression.html
- Parâmetros da Decision Tree Regression: http://scikit-learn.org/stable/modules/generated/sklearn.tree.DecisionTreeRegressor.html
- ForestCI:
- Docs: http://contrib.scikit-learn.org/forest-confidence-interval/reference/forestci.html
- Repositório github: https://github.com/scikit-learn-contrib/forest-confidence-interval
- Conectando ao SQL Server:
- https://docs.microsoft.com/en-us/sql/connect/python/pyodbc/step-3-proof-of-concept-connecting-to-sql-using-pyodbc
- https://stackoverflow.com/questions/33725862/connecting-to-microsoft-sql-server-using-python
- https://stackoverflow.com/questions/38534154/linux-python3-cant-open-lib-sql-server
- https://docs.microsoft.com/en-us/sql/connect/odbc/linux-mac/installing-the-microsoft-odbc-driver-for-sql-server
Miscelâneas
- Constantes físicas, matemáticas e unidades no Scipy: http://docs.scipy.org/doc/scipy/reference/constants.html
- Fazendo matemática simbólica no python: http://scipy-lectures.github.io/advanced/sympy.html
- Handbook of the Physics Computing Course: http://pentangle.net/python/handbook/
- Lista de diversas bibliotecas, módulos e pacotes de cunho numérico e científico: https://wiki.python.org/moin/NumericAndScientific
- Abramowitz and Stegun. Handbook of Mathematical Functions.http://people.math.sfu.ca/~cbm/aands/toc.htm
- Trabalhando com fasores no python: http://csserver.evansville.edu/~richardson/courses/Tutorials/complex_numbers/python/python.pdf
- Biblioteca numérica de precisão arbitrária para Python http://docs.sympy.org/dev/modules/mpmath/index.html
- Notas no uso científico do Python por Scott Sievert - Menos completa que esta, entretanto, mais detalhada: http://scottsievert.github.io/blog/2014/05/14/Scientific-Python-tips-and-tricks/
- Primeiros passos raṕidos ipython + numpy: http://www.linuxuser.co.uk/tutorials/numpy-python-tutorial
- Compilando/Congelando seu código em python: http://docs.python-guide.org/en/latest/shipping/freezing/
- pyinstaller: https://pythonhosted.org/PyInstaller/
- Executando scripts python dentro do iPython e salvando as variáveis para uso interativo: http://ipython.readthedocs.org/en/latest/interactive/magics.html#magic-run
- Empurrando as variáveis da main(), ou qualquer outra função, para a shell do iPython: http://ipython.readthedocs.org/en/stable/api/generated/IPython.core.interactiveshell.html#IPython.core.interactiveshell.InteractiveShell.push http://ipython.readthedocs.org/en/stable/api/generated/IPython.core.getipython.html (thanks nbastin -> #ipython @ freenode)
- Creating virtual enviroments: http://docs.python-guide.org/en/latest/dev/virtualenvs/ - Great for remote computers without root access!
- Python packaging:
- https://medium.com/small-things-about-python/lets-talk-about-python-packaging-6d84b81f1bb5#.k734sqxuu
- https://pypi.python.org/pypi?%3Aaction=list_classifiers
- https://packaging.python.org/distributing/
- http://stackoverflow.com/questions/1550226/python-setup-py-uninstall
- Fifteen Python libraries for data science: http://www.kdnuggets.com/2017/06/top-15-python-libraries-data-science.html#.WZB6larsoLA.twitter
Aprendendo Python
- Tutorial interativo web, ótimo início:http://www.learnpython.org/
- Ótima documentação:http://learnpythonthehardway.org/book/
- Dive Into Python (A referência mais usadas por programadores python conhecidos: http://www.diveinto.org/python3/
- Python for Bash scripters:A well-kept secret: http://magazine.redhat.com/2008/02/07/python-for-bash-scripters-a-well-kept-secret/
- Dealing with tuples: http://www.tutorialspoint.com/python/python_tuples.htm
- Dictionaries basic: http://www.tutorialspoint.com/python/dictionary_get.htm
- GUI:
- Tkinter 8.5 reference: a GUI for Python http://infohost.nmt.edu/tcc/help/pubs/tkinter/web/index.html
- Basic debugging with pdb: https://pythonconquerstheuniverse.wordpress.com/2009/09/10/debugging-in-python/