Posts

Showing posts with the label Performance

ANALYZE TABLE..VALIDATE STRUCTURE Runs Forever

Answer : UPDATE: What Worked... So after reading the link from @Raj and reading @ora-600's answer I tried to validate the database with the RMAN command backup check logical validate database; . While this worked fine, it was also clear that it was not looking at everything that the ANALYZE INDEX command would. After trying many different variations, I finally discovered that this command would work: ANALYZE TABLE .. VALIDATE STRUCTURE CASCADE offline; Yep, just switching to OFFLINE appears to fix it. And that eventually led me to this bug# 5752105 on this page: http://www.eygle.com/case/10204_buglist.htm. I am not in a position to prove it right now (cannot apply any patches for the time being), but I strongly suspect that this is what I was running into. So while the question is not fully answered, I am going to mark @ora-600's very helpful answer as correct so that he can collect Paul White's very generous bounty. I think the article Raj quoted (https:...

Can I Profile NodeJS Applications Using Visual Studio Code?

Image
Answer : There is no plugin/support that I am aware of for initiating the profiling, or heap dumps, etc that are available in Chrome's Dev Tools. But, the VS Code debugger can work alongside the dev tools debugger. Start from VS Code, and start debugging as you would. Then open dev tools from any tab in the Chrome/Chromium browser, and look for the green icon indicating a node.js debugging process is running ( manually done via node --inspect ): . Click this green icon and you'll have many of the browser dev tools features for debugging the node.js process, specifically the memory and profiler tabs. Visual Studio Code 1.45 (April 2020) should help, as it integrates Javascript debugging capabilities, including profiling: New JavaScript debugger This month we've continued making progress on our new JavaScript debugger. It's installed by default on Insiders, and can be installed from the Marketplace in VS Code Stable. You can start using it with your e...

Case-insensitive String Startswith In Python

Answer : You could use a regular expression as follows: In [33]: bool(re.match('he', 'Hello', re.I)) Out[33]: True In [34]: bool(re.match('el', 'Hello', re.I)) Out[34]: False On a 2000-character string this is about 20x times faster than lower() : In [38]: s = 'A' * 2000 In [39]: %timeit s.lower().startswith('he') 10000 loops, best of 3: 41.3 us per loop In [40]: %timeit bool(re.match('el', s, re.I)) 100000 loops, best of 3: 2.06 us per loop If you are matching the same prefix repeatedly, pre-compiling the regex can make a large difference: In [41]: p = re.compile('he', re.I) In [42]: %timeit p.match(s) 1000000 loops, best of 3: 351 ns per loop For short prefixes, slicing the prefix out of the string before converting it to lowercase could be even faster: In [43]: %timeit s[:2].lower() == 'he' 1000000 loops, best of 3: 287 ns per loop Relative timings of these approaches will of course depe...

Best Way To Select Random Rows PostgreSQL

Answer : Given your specifications (plus additional info in the comments), You have a numeric ID column (integer numbers) with only few (or moderately few) gaps. Obviously no or few write operations. Your ID column has to be indexed! A primary key serves nicely. The query below does not need a sequential scan of the big table, only an index scan. First, get estimates for the main query: SELECT count(*) AS ct -- optional , min(id) AS min_id , max(id) AS max_id , max(id) - min(id) AS id_span FROM big; The only possibly expensive part is the count(*) (for huge tables). Given above specifications, you don't need it. An estimate will do just fine, available at almost no cost (detailed explanation here): SELECT reltuples AS ct FROM pg_class WHERE oid = 'schema_name.big'::regclass; As long as ct isn't much smaller than id_span , the query will outperform other approaches. WITH params AS ( SELECT 1 AS min_id...

Can Google.com And Other Heavily Trafficked Websites Get A "fast" Rank Using Google's PSI API?

Image
Answer : To directly answer the question, no it's not impossible to get a fast FCP label. There's more to the question so I'll try to elaborate. Another way to phrase the "fast" criteria is: "Do at least 90% of user experiences have an FCP less than 1 second ?" Why 90%? Because it's inclusive of a huge proportion of user experiences. As the PSI docs say: Our goal is to make sure that pages work well for the majority of users. By focusing on 90th and 95th percentile values for our metrics, this ensures that pages meet a minimum standard of performance under the most difficult device and network conditions. Why 1 second? It's a subjective value for how quickly users expect the page to start showing meaningful progress. After 1 second, users may become distracted or even frustrated. Of course the holy grail is to have instant loading, but this is chosen as a realistic benchmark to strive towards. So at worst 10% of the FCP experien...

Apache Spark: Map Vs MapPartitions?

Answer : Imp. TIP : Whenever you have heavyweight initialization that should be done once for many RDD elements rather than once per RDD element, and if this initialization, such as creation of objects from a third-party library, cannot be serialized (so that Spark can transmit it across the cluster to the worker nodes), use mapPartitions() instead of map() . mapPartitions() provides for the initialization to be done once per worker task/thread/partition instead of once per RDD data element for example : see below. val newRd = myRdd.mapPartitions(partition => { val connection = new DbConnection /*creates a db connection per partition*/ val newPartition = partition.map(record => { readMatchingFromDB(record, connection) }).toList // consumes the iterator, thus calls readMatchingFromDB connection.close() // close dbconnection here newPartition.iterator // create a new iterator }) Q2. does flatMap behave like map or like mapPart...

Bitlocker Performance Impact On SSD

Image
Answer : You should have a negligible performance impact with most SSDs. Especially with the latest Intel CPUs that can do hardware AES way faster than a drive (any drive) can read or write. My MacBook Pro pushes over 900 megabytes per second with AES according to the TrueCrypt benchmark, and that's a laptop. On my desktop I use 4 Samsung SSDs in RAID0 and I have BitLocker turned on. TrueCrypt on this same machine reports over 5GB/sec for AES. (Two 6-core Xeons...) That said, the SandForce SSD controller is said to do some internal compression/dedupe (which was proven via benchmarks that used large compressed files that it could not "optimize"). Obviously this is not going to work at all with BitLocker where every encrypted sector will be completely unique and uncompressible. So if you're planning on using an SSD, don't get a SandForce one - or if you do, make sure you can return it if you find that performance really degrades after you turn BitLocker on. ...

Apple - Can I Manually Limit The %CPU Used By A Process?

Answer : cputhrottle is the tool you need. You can install it with Homebrew. You can monitor a series of processes by name by running the Bash script below. I'm not quite sure how to turn this into a login item since cputhrottle requires superuser permissions. Run it as a script, in an Automator workflow, whatever: # Get the Process/App names from Activity Monitor and put them here apps=("AppOne" "AppTwo" "AppThree") # Set the respective limits here limits={30 40 50) while true; do for app in ${apps}; do for limit in ${limits}; do for pid in $(pgrep ${app}); do sudo /path/to/cputhrottle ${pid} ${limit} done done done done [Edited] I've added a different version for this script (a bash script), which might be useful for people looking for limiting the CPU for multiple applications. This new script also allows you to specify a list containing the application name and the CPU limit for it. The main di...

Can Jconsole Data Be Retrieved From The Command Line?

Answer : jconsole just provides a wrapper around the JMX MBeans that are in the platform MBeanServer . You can write a program to connect to your VM using the Attach API which would then query the MBeans. Or you can expose the platform MBeanServer over RMI and query the MBeans that way. See the java.lang.management package for more info Maybe jvmtop is worth a look. It's a command-line tool which provides a live-view for several metrics. Example output of the VM overview mode: JvmTop 0.4.1 amd64 8 cpus, Linux 2.6.32-27, load avg 0.12 http://code.google.com/p/jvmtop PID MAIN-CLASS HPCUR HPMAX NHCUR NHMAX CPU GC VM USERNAME #T DL 3370 rapperSimpleApp 165m 455m 109m 176m 0.12% 0.00% S6U37 web 21 11272 ver.resin.Resin [ERROR: Could not attach to VM] 27338 WatchdogManager 11m 28m 23m 130m 0.00% 0.00% S6U37 web 31 19187 m.jvmtop.JvmTop 20m 3544m 13m 130m 0.93% 0.47% S6U37 web 20 16733 artup.Bootstrap 159m...