Monthly Archives: August 2024

Statistics and probability

Resources on Advanced Statistics & Probability as per ChatGPT

Leave a comment

Here’s a list of some excellent resources across various formats—books, YouTube channels, and blogs—that cover advanced statistics and probability theories:

Books:

1. “Introduction to Probability” by Dimitri P. Bertsekas and John N. Tsitsiklis

– A comprehensive introduction to probability, available for free in PDF form on MIT’s OpenCourseWare.

– [Link to PDF](https://athenasc.com/probbook.html)

2. “Think Stats” by Allen B. Downey

– This book focuses on applying statistics to real-world data, with practical examples using Python.

– [Link to PDF](https://greenteapress.com/wp/think-stats-2e/)

3. “The Elements of Statistical Learning” by Trevor Hastie, Robert Tibshirani, and Jerome Friedman

– A highly regarded book on statistical learning, offering a deep dive into many advanced topics.

– [Link to PDF](https://web.stanford.edu/~hastie/ElemStatLearn/)

4. “All of Statistics: A Concise Course in Statistical Inference” by Larry Wasserman

– A great resource that covers both basic and advanced topics in statistics.

– [Link to PDF](https://www.stat.cmu.edu/~larry/all-of-statistics/)

5. “Introduction to Statistical Thought” by Michael Lavine

– This book covers the foundations of statistical inference and is available for free online.

– [Link to PDF](https://www.math.umass.edu/~lavine/Book/book.html)

YouTube Channels:

1. MIT OpenCourseWare – Probability and Statistics

– Features lectures from MIT’s undergraduate and graduate courses, including advanced topics in probability and statistics.

– [MIT OCW YouTube Channel](https://www.youtube.com/user/MIT)

2. Khan Academy

– Although mostly known for basic statistics, Khan Academy also offers more advanced courses in probability and statistical inference.

– [Khan Academy – Probability & Statistics](https://www.youtube.com/user/khanacademy)

3. StatQuest with Josh Starmer

– Excellent channel that breaks down complex statistical concepts into easily understandable segments, including advanced topics.

– [StatQuest YouTube Channel](https://www.youtube.com/user/joshstarmer)

4. Brilliant.org

– While Brilliant offers paid content, their YouTube channel provides free videos covering advanced mathematical concepts, including probability and statistics.

– [Brilliant.org YouTube Channel](https://www.youtube.com/c/Brilliantorg)

5. Harvard University – STAT110 (Probability)

– Lectures from Harvard’s popular STAT110 course, taught by Professor Joe Blitzstein, covering probability theory in depth.

– [Harvard University – STAT110 YouTube Channel](https://www.youtube.com/playlist?list=PL2SOU6wwxB0v1kQTpqpuuGIjJRWJaFfeH)

Blogs and Online Courses:

1. Cross Validated (Stack Exchange)

– A Q&A site specifically for statistics, probability, and data science. It’s a great place to see advanced problems discussed in depth.

– [Cross Validated](https://stats.stackexchange.com/)

2. OpenIntro

– Offers free textbooks, labs, and resources on statistics, including advanced topics.

– [OpenIntro](https://www.openintro.org/)

3. Towards Data Science (Medium)

– A popular blog on Medium with numerous articles on advanced statistics, probability, and their applications in data science.

– [Towards Data Science](https://towardsdatascience.com/)

4. DataCamp Community

– While DataCamp offers paid courses, their blog has free articles and tutorials on advanced statistical methods.

– [DataCamp Community](https://www.datacamp.com/community)

5. Probability and Statistics EBook

– An online resource that provides detailed explanations of advanced topics in probability and statistics.

– [Probability and Statistics EBook](http://www.probabilitycourse.com/)

MOOCs and Online Lectures:

1. Coursera – Statistical Learning by Stanford University

– A free course that covers statistical learning, based on the book “The Elements of Statistical Learning.”

– [Coursera – Statistical Learning](https://www.coursera.org/learn/statistical-learning)

2. edX – Probability: The Science of Uncertainty and Data by MIT

– A free course that dives deep into probability theory and applications.

– [edX – MIT Probability Course](https://www.edx.org/course/probability-the-science-of-uncertainty-and-data)

3. Harvard Online Learning – Data Science: Probability

– Part of Harvard’s Data Science Professional Certificate, this course is available for free auditing.

– [Harvard – Data Science: Probability](https://online-learning.harvard.edu/course/data-science-probability)

These resources cover a broad range of advanced topics in probability and statistics, and they offer various levels of depth, from introductory overviews to rigorous academic treatments.

Liferay, Search

How to tame the SEO beast with Liferay? Part 1.

Leave a comment

Here are some keywords and concepts to explore:

1. Performance tuning – https://www.linkedin.com/pulse/performance-tuning-liferay-part-3-neil-harwani-nsoof/

2. Performance options for pages & search in built in Liferay – -> https://learn.liferay.com/w/dxp/using-search/search-pages-and-widgets/search-insights

–> https://learn.liferay.com/w/dxp/content-authoring-and-management/page-performance-and-accessibility/analyze-seo-and-accessibility-on-pages

–> https://learn.liferay.com/w/dxp/content-authoring-and-management/page-performance-and-accessibility/about-the-page-audit-tool

3. SEO features in Liferay

–> https://learn.liferay.com/w/dxp/site-building/optimizing-sites

–> https://learn.liferay.com/w/dxp/site-building/displaying-content/using-display-page-templates/configuring-seo-and-open-graph

4. Set up your own monitoring (simple JMeter is a good start) and focus on page load times in conjunction with other things rather than only scores. Use various tools – an example: –> https://bloggerspassion.com/website-performance-speed-test-tools/

–> https://developer.chrome.com/docs/lighthouse/overview/

5. Liferay headless – https://www.liferay.com/solutions/headless-apis

6. Lazy loading, innovative solutions like lighter pages with type ahead and so on

7. Understand various tools for website performance, formula for page speed insights & SEO but focus on your metrics like page load speed, image quality. Don’t blindly pick a tool and follow it. However, suggestions / recommendations / insights of various tools should be explored and worked upon as needed

8. Note: Mobile score is throttled down by page speed insights to slow 3G or so

Other hints:

1. Robots.txt

2. Core web vitals

3. Sitemap

4. https

5. Broken links, friendly URLs, mobile friendliness

6. Content quality

7. Tags

8. Images

9. Caching

Liferay with right configuration, customization & tuning is capable of some magical things 🙂

FPM

Keywords from my FPM journey – Part 2

Leave a comment

Some of the keywords from my FPM journey – Part 2:

Analytics, Data Science

Learnings from eMasters Data Science for Decision Making – IIT Gandhinagar – Part 1

Leave a comment

Below are the learnings from eMasters Data Science for Decision Making – IIT Gandhinagar – Part 1. This needs to be an article in itself. You can find more details here about eDSDM here: e-Masters | IITGNX

  • There is Mathematics for your AIML problem in feature engineering, preprocessing, evaluation metrics, models, errors, etc. and so on or you can build some relationship between Mathematics and your problem either by breaking the problem into pieces or transforming data and so on.
  • Human intuition is still invaluable like in situations of imbalanced datasets, regression and more.
  • Visualization and EDA almost always help for your problems. For higher dimension problems you can do PCA, T-SNE, shadowing on lower dimensions, etc. more approaches to bring it to lower dimensions with or without transformation like basis transformation. In this situation, you should also feed data after this transformation for testing / using the model.
  • There are problems beyond model selection and preprocessing, feature engineering like data quality, overfitting, evaluation, errors, hyper parameter tuning and generalization which we need to think about.
  • Situations like classification of medical diagnosis issues have very high value repercussions for wrong classification whereas some models with overfitting won’t generalize and cause problems like in normal situations with regression.
  • Probability concepts like equally likely for randomness, distributions, fairness, probability tree, joint probability, permutations and combinations, etc. have lot of value and are worth learning.
  • There is no learning like learning by experience, concepts and application.
  • Your assumptions maybe wrong as well so it’s good to verify.
  • Accuracy alone may not be a good measure, please add recall & precision as well in your analysis especially in imbalanced datasets.
  • Domain knowledge matters. Don’t ignore this.
  • Email me: Neil@HarwaniSytems.in
  • Website: www.HarwaniSystems.in
  • Blog: www.TechAndTrain.com/blog
  • LinkedIn: Neil Harwani | LinkedIn
Academics

Returning to school / academics from industry

Leave a comment

Below is an article summarizing some points that I have experienced when transitioning back to academics / school from industry. I have done academics (learning) part time which is study part time since 2011 onwards to achieve various goals in academics along with work in industry. Here is the summary of what it takes and some tips to excel:

  • You need to accept that getting a degree or a good certification takes time and effort.
  • You need dedicated time over the nights or mornings on weekdays and especially half of the weekends sacrificing time with family and friends.
  • Calendaring or scheduling time using calendars is your best friend.
  • Finding out the best resources from the internet and Wikipedia or similar portals is very helpful.
  • You can do anything but not everything. This is actually true. You need to drop / deprioritize what you cannot do due to lack of time.
  • Your industry & family environment needs to be supportive of your goals and efforts, only then you will be able to manage both industry and academics.
  • Writing / journaling also definitely helps, something like a blog as well can help.
  • Pick growth mindset, have an open mind and learn continuously. This needs to become a habit.
  • Use your industry knowledge to have discussions with batchmates/peers and professors. This helps to learn quickly and have engaging discussions.
  • Integrate industry practices into your academic work.
  • Network with industry, professors, batchmates to learn more effectively and stay on top of trends.
  • Take advantage of academic assets like libraries and online databases for research.
  • Use online platforms for research, collaboration, and project management.
  • Understand mixing theory and practice. It helps.
  • Aim to bridge the gap between academia and industry in your work.
  • Plan your savings and finances to manage academic expenses properly.
  • Maintain a healthy balance between work, life and study time.
  • Manage stress through exercise, proper nutrition, and mindfulness practices.
  • Set clear, achievable, planned goals and not ad-hoc random expectations. Adjust as necessary.
  • Be flexible to new academic environments.
  • Try for innovation using your industry and academic knowledge.
  • Returning to good academic degrees / diplomas / certifications / workshops will most likely improve your knowledge and skills significantly.
  • Email me: Neil@HarwaniSytems.in
  • Website: www.HarwaniSystems.in
  • Blog: www.TechAndTrain.com/blog
  • LinkedIn: Neil Harwani | LinkedIn